{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第一个线性感知机算法：PLA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 1. 什么是感知机（Perceptron）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**PLA**全称是Perceptron Linear Algorithm，即线性感知机算法，属于一种最简单的**感知机**（Perceptron）模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "感知机模型是机器学习二分类问题中的一个非常简单的模型。它的基本结构如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/1.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$x_i$是输入，$w_i$表示**权重系数**，$b$表示**偏移常数**。感知机的线性输出为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$scores=\\sum_i^Nw_ix_i+b$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了简化计算，通常我们将$b$作为**权重系数**的一个维度，即$w_0$。同时，将输入$x$扩展一个维度，为1。这样，上式简化为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$scores=\\sum_i^{N+1}w_ix_i$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$scores$是感知机的输出，接下来就要对$scores$进行判断："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 若$scores\\geq0$，则$\\hat y=1$（正类）\n",
    "\n",
    "* 若$scores<0$，则$\\hat y=-1$（负类）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是线性感知机模型的基本概念，简单来说，它由**线性得分计算**和**阈值比较**两个过程组成，最后根据比较结果判断样本属于正类还是负类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. PLA理论解释"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于二分类问题，可以使用感知机模型来解决。PLA的基本原理就是**逐点修正**，首先在超平面上随意取一条分类面，统计分类错误的点；然后随机对某个错误点就行修正，即变换直线的位置，使该错误点得以修正；接着再随机选择一个错误点进行纠正，分类面不断变化，直到所有的点都完全分类正确了，就得到了最佳的分类面。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用二维平面例子来进行解释，第一种情况是错误地将正样本（y=1）分类为负样本（y=-1）。此时，$wx<0$，即$w$与$x$的夹角大于90度，分类线$l$的两侧。修正的方法是让夹角变小，修正$w$值，使二者位于直线同侧："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$w:=w+x=w+yx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "修正过程示意图如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/2.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "第二种情况是错误地将负样本（y=-1）分类为正样本（y=1）。此时，$wx>0$，即$w$与$x$的夹角小于90度，分类线$l$的同一侧。修正的方法是让夹角变大，修正$w$值，使二者位于直线两侧："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$w:=w-x=w+yx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "修正过程示意图如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/3.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "经过两种情况分析，我们发现PLA每次$w$的更新表达式都是一样的：$w:=w+yx$。掌握了每次$w$的优化表达式，那么PLA就能不断地将所有错误的分类样本纠正并分类正确。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集存放在'../data/'目录下，该数据集包含了100个样本，正负样本各50，特征维度为2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv('./data/data1.csv', header=None)\n",
    "# 样本输入，维度（100，2）\n",
    "X = data.iloc[:,:2].values\n",
    "# 样本输出，维度（100，）\n",
    "y = data.iloc[:,2].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据分类与可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们在二维平面上绘出正负样本的分布情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEWCAYAAABliCz2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUHGW57/Hvk0nC5Ea4JAoSkxE3\nXkJMhmSScMtFgQhe8IIiGBcHj+ywuQ6s4BEMkKgb9vHIMqAckIjHoztxi3I7yiKA6EYDBHXCRRA4\nwlaCObAliSRAhsBcnvNHdc90d3pmqnq6uqu6fp+1es1UdXXVW1XJM9XPU+9b5u6IiEjjG1HvBoiI\nSG0o4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6knpl92cxurPayIdblZvYP1ViXSC2Y7sOX\nJDGz04HlwDuBV4DbgEvcfUc921WOmTlwiLs/W+a9+4DDgS7AgWeAnwKr3f2N4a5fpBK6wpfEMLPl\nwNeBLwITCQLmNOAXZjZ6gM+MrF0LIzvX3ScABxL8ETsFuNPMrL7NkqxSwJdEMLO9ga8A57n7Xe7e\n5e7PAScTBP3P5ZZbZWY3m9laM3sFOD03b23Buk4zs81mtt3MLjOz58zs2ILPr8393pJLy/wXM3ve\nzLaZ2YqC9cwzs41mtsPMXjSzawf6wzMYd9/l7vcBJwJHAB8eav1m9pvcxx8zs9fM7DNmtq+Z3WFm\nW83s5dzvU6K2R7JLAV+S4kigGbi1cKa7vwasB44rmP0x4GZgH2Bd4fJmNh24DlhKcGU9EThoiG0f\nDbwbOAa43Mzem5vfA1wITCII1McAZ0fcr8J9eR7oABYMtX53X5hbZpa7j3f3mwj+v36f4A/gVOB1\n4NpK2yPZo4AvSTEJ2Obu3WXeezH3ft5Gd7/d3Xvd/fWSZT8F/Nzd73f3N4HLCXLog/mKu7/u7o8B\njwGzANx9k7s/5O7duW8bNwCLou9akReA/SpZv7tvd/db3L3T3V8FrqhCeyRDkpz/lGzZBkwys5Fl\ngv6Buffz/jrIet5W+L67d5rZ9iG2/Z8Fv3cC4wHM7F3AN4E2YCzB/5dNQ6xrKAcBD1ayfjMbC6wG\njgf2zc2eYGZN7t4zzHZJBugKX5JiI/AG8MnCmWY2DjgB+GXB7MGu2F8E+vLaZjYG2L/CNl0PPE1w\np8zewJeBiguuZvZ2YA6wocL1LydIPc3PLZ9P+6gILKEo4EsiuPtOgqLtt83seDMbZWYtBLcybgH+\nNeSqbgY+amZH5gqgX6HygDiB4NbQ18zsPcBZlazEzMaa2SLg/wC/A+4Muf6/AQeXtOd1YIeZ7Qes\nrKQ9kl0K+JIY7v4/CK5yryIIhL8lSM8cE/bedXf/I3Ae8GOCq/1XgZcIvj1EdRHw2dw6vgvcFPHz\n15rZqwSB+2rgFuB4d+8Nuf5VwA9yd/GcnFvHGIL01kPAXVF3SLJNHa+koZnZeGAHQdrkL/Vuj0g9\n6QpfGo6ZfTSXRhlH8G3hceC5+rZKpP4U8KURfYzg9scXgEOAU1xfZUWU0hERyQpd4YuIZESiOl5N\nmjTJW1pa6t0MEZHU2LRp0zZ3nxxm2UQF/JaWFjo6OurdDBGR1DCzzWGXVUpHRCQjFPBFRDJCAV9E\nJCMSlcMvp6uriy1btrB79+56NyV1mpubmTJlCqNGjap3U0QkARIf8Lds2cKECRNoaWlBT4YLz93Z\nvn07W7Zs4R3veEe9myMiCZD4lM7u3bvZf//9FewjMjP2339/fTMSkT6JD/iAgn2FdNwa37p10NIC\nI0YEP9etG+oTkmWJT+mISHnr1sGyZdDZGUxv3hxMAyxdWr92SXKl4gq/3pqammhtbWXGjBl8+tOf\npjP/PyyCM844gyeffBKAK6+8sui9I488sirtlGxZsaI/2Od1dgbzRcpRwA9hzJgxPProozzxxBOM\nHj2a73znO5HXceONNzJ9+nRgz4D/4IMPVqWdki3PPx9tvkjDBfy4c5oLFizg2WefBeCb3/wmM2bM\nYMaMGVx99dUA7Nq1iw9/+MPMmjWLGTNmcNNNwUOMFi9eTEdHBxdffDGvv/46ra2tLM197x4/fjwA\nn/nMZ7jzzjv7tnX66adzyy230NPTwxe/+EXmzp3LzJkzueGGG6q7U5JKU6dGmy+Cu8f2InjoxOPA\no0DHUMvPmTPHSz355JN7zBvI2rXuY8e6Q/9r7Nhg/nCMGzfO3d27urr8xBNP9Ouuu847Ojp8xowZ\n/tprr/mrr77q06dP94cffthvvvlmP+OMM/o+u2PHDnd3X7Rokf/+978vWl/p+m+99VY/7bTT3N39\njTfe8ClTpnhnZ6ffcMMN/rWvfc3d3Xfv3u1z5szxP//5z6HaHuX4SbrE9e89C9audZ82zd0s+Jnm\nYxYmtuZftbjCf7+7t7p7W9wbiiunmb8ib2trY+rUqXzhC1/g/vvv5xOf+ATjxo1j/PjxfPKTn2TD\nhg28733v49577+VLX/oSGzZsYOLEiaG3c8IJJ/CrX/2KN954g/Xr17Nw4ULGjBnDPffcww9/+ENa\nW1uZP38+27dv55lnnhneTknqLV0Ka9bAtGlgFvxcs0YF26Hki92bNwd/JvPF7izc4dRQd+nEldPM\n5/AL+QAPjnnXu97Fpk2buPPOO7nkkktYsmQJl19+eajtNDc3s3jxYu6++25uuukmTj311L5tffvb\n3+aDH/zg8HZEGs7SpQrwUQ12YdjoxzLuK3wH7jGzTWa2rNwCZrbMzDrMrGPr1q3D2lgtc5oLFy7k\n9ttvp7Ozk127dnHbbbexYMECXnjhBcaOHcvnPvc5LrroIh5++OE9Pjtq1Ci6urrKrveUU07h+9//\nPhs2bOgL8B/84Ae5/vrr+z7zpz/9iV27dlV/p0QyIMvF7rgD/lHuPhs4ATjHzBaWLuDua9y9zd3b\nJk8ONYb/gK64AsaOLZ43dmwwv9pmz57N6aefzrx585g/fz5nnHEGhx12GI8//jjz5s2jtbWVK664\ngksvvXSPzy5btoyZM2f2FW0LLVmyhN/85jcce+yxjB49Gghu6Zw+fTqzZ89mxowZnHnmmXR3d1d/\np0QqkLbOX3FeGCb+WIRN9g/3BawCLhpsmeEWbd0bqxhTDSraSpzSWDiOq831OhYkoWhrZuPMbEL+\nd2AJ8ERc28tbuhSeew56e4OfjZ6TE6mnNHb+iqvYnYZjEWfR9q3AbbnxXEYCP3L3u2LcnojUWFrz\n4XEUu9NwLGIL+O7+Z2BWXOsXkfqbOjW4rbHc/KxJw7FouJ62ImmX+MJfgVreKFEvYc9HGo6FAr5I\ngqStU1Cjd/6Kcj7ScCzMB+hAVA9tbW3e0dFRNO+pp57ive99b51alH46funS0lI+LTBtWnATgtRW\nGs6HmW3ykCMZ6Ao/BDNj+fLlfdNXXXUVq1atqvp2NGyypKHwlyWNdj4aL+CXfmOpwjeYvfbai1tv\nvZVt27YNe12D0bDJohEwk6XRzkdjBfxVq+DCC/uDvHswPcyr8ZEjR7Js2TJWr169x3tbt27lpJNO\nYu7cucydO5cHHnigb/5xxx3H7NmzOfPMM5k2bVrfH4yPf/zjzJkzh0MPPZQ1a9YAaNhkAeIt/J19\nNowcGeSXR44MpqshTUXmvCQUYuty3ML20KrFa1g9bXt73dvbg+5t7e3lpys0btw437lzp0+bNs13\n7Njh3/jGN3zlypXu7n7qqaf6hg0b3N198+bN/p73vMfd3c855xy/8sor3d19/fr1DvjWrVvd3X37\n9u3u7t7Z2emHHnqob9u2rW87pdt1H96wyeppmz5x9BY/66ziHqD511lnDb+tjd7TNo7zUc3jRoSe\ntnUP8oWvYQ+tUBjk869hBnv3/sB72WWX+Ve/+tWigD958mSfNWtW3+ttb3ubv/LKKz5r1qyi4Lvv\nvvv2BfyVK1f6zJkzfebMmb733nv7xo0bi7ZTut3XX3/dp0yZ4rt37/bbb7/dP/vZz7q7+0knneSH\nHHJI37ZbWlr87rvvLlqHAr64uzc1lQ/4TU3DW++0aeXXO21aNVodjyS0uZptiBLwG2p4ZMxg9Wq4\n5pr+eatXB/Or4IILLmD27Nl8/vOf75vX29vLxo0bGTNmTNGywXnY03333ce9997Lxo0bGTt2LIsX\nL2b37t2DblfDJstw9fREmx9WGouaSWhzvdrQWDl8z+XsCxXm9Idpv/324+STT+Z73/te37wlS5Zw\n7bXX9k3nx80/+uij+clPfgLAPffcw8svvwzAzp072XfffRk7dixPP/00Dz30UN9nNWyyxKWpKdr8\nsNJY1ExCm+vVhsYJ+Plgf8010N4ejJ7W3h5MVzHoL1++vOhunW9961t0dHQwc+ZMpk+f3veA85Ur\nV3LPPfcwe/Zs1q9fz4EHHsiECRM4/vjj6e7uZubMmVx22WUcfvjhfevSsMkSl2Vln0Yx8PywklTU\nTEIhNqy6tSFs7qcWr2Hn8FeuLM7Z53P6uXx7Le3evdu7urrc3f3BBx/0WbNm1bwN7srhS2DtWveR\nI4vzxSNHVq8AWe+iZhIKsVFVqw1EyOE3Xk9b9+Kcfel0jTzzzDOcfPLJ9Pb2Mnr0aK677jrmzp1b\n83aop61AOnqMFora3rTtXzVF6WnbWEVb2DO41yHYAxxyyCE88sgjddm2SKkkFCqjiNretO1fvaQi\nh5+kbyFpouMmeUkoVEYRtb1p2796SXzAb25uZvv27QpeEbk727dvp7m5ud5Nqbq4eihGWW9SepeG\n7T0btUhY72Mctb1JKMSmQeJTOlOmTGHLli1s3bq13k1JnebmZqZMmVLvZlRVfrja/KPk8sPVwvCG\noY2y3rjaENXZZ8P11/dP9/T0T193XfGy+XatWBGkOaZODYJhufYm4RhHaW8ly2dV4ou2IoXiKs5F\nWW9SCoQjR5bvONXUBMO5OzcJx1jCi1K0VcCXVBkxonyXCrOg60Ut1htXG6Ia7H6E4fy3TsIxlvA0\nHr40rKjFubA54yjrTUqBMCm9Z+M4xpVIYw2m5sLesF+LV7mOVyKFonSwScKycUrCCJhJOW5JaUc9\n0EijZYqUCttDMeqIhFF6Piahp6Z7ENzzI2E2NQ0/2Ocl4RhHEaUdSRgts5qiBHzl8KVhKWccv6Qc\n4zTWYKpFOXwRkpNrb2RJOcZprMHUgwK+NCx1xqlcXB2k4iqWRmlHpv9dhM391OKlHL5UW1Jy7WkS\n18iTcRdL01iDqQaUwxeRSqnjVboohy8iFYtr5EmNaFl/CvgiUiSuomaWi6VJoYAvkhOloBh2lMq4\n2xGHuIqamS6WJkXYZH8tXiraSr1EKSjG1cM1ajviFFdRs5GKpUmBirYi0UQpKMY1SmXUdoiAirYi\nkUUpKJYL9oPNj6sdIlEp4IsQraAY1yiVUdshEpUCvghB4XD06OJ5o0eXLyjmn9IUdn6UIqwKmxIn\nBXyRnNJy1kDlraOOCvL4hUaODOaXyj/Wb/PmYH35x/oNFPSXLoU1a4KcvVnwc80aPapPqkNFWxHi\ne8ShirASNxVtRSKKUiyNa1mRuMUe8M2sycweMbM74t6WSKmwHaTiGl5Xj/WTJKnFFX478FQNtiNS\n5Oyz4frr+2+X7OkJpssF/biG1/3Qh8q3baD5UUSpD0StJUiDCttDq5IXMAX4JfAB4I6hlldPW6mm\n/KP/Sl9NTeWXj2N43Tgfp5flx/pJP5LS09bMbgb+BZgAXOTuHymzzDJgGcDUqVPnbC5X4RKpgNnA\n79XqXoU4H6eX5cf6Sb9EFG3N7CPAS+6+abDl3H2Nu7e5e9vkyZPjao5kUJwdpMKKM4efpFqCpEOc\nOfyjgBPN7Dngx8AHzGxtjNuTBElCgTDODlJhRenQVcm69Vg/iSRs7mc4L2AxyuFnRlJGfHQPRrDM\n5/KbmgYe0TKuNq9d6z5qVPF6R43SY/2kekhKDj/PzBYzQA6/kDpeNYY0djbSY/0kraLk8NXTVqou\njQXCuNqcxmMh6ZKIoq1kVxoLhHqsn2SBAr5UXZIKhGELsXqsn2RC2GR/LV4q2jaOJBQIoxZi9Vg/\nSSOSVrQNSzl8qSYVTCULlMMXQSNVipRSwJeGpYKpSDEFfGlYV1wBo0YVzxs1SgVTyS4FfGlopQOo\nDTagmkijU8CXhrViBbz5ZvG8N98M5otkkQK+NCwVbUWKKeBLw4patE3CCJ8icVLAl4YVpZerHgEo\nWaCALw1r6VJYsyboaGUW/FyzJphfasUK6OwsntfZqXy/NBb1tBVBo1pKeqmnrUhE6qQlWaCAn3Eq\nVAaijmqp4yZpNLLeDZD6yRcq87nrfKESyue5G1l+f1esCG7bnDo1CPbljoOOm6SVcvgZptEkK6Pj\nJkmiHL6Eoo5JldFxk7QaMOCb2fvM7CEz+6uZrTGzfQve+11tmidxUqGyMjpuklaDXeFfD6wC3gf8\nCbjfzN6Ze2/UQB+S9NDj9yqj4yZpNVjAH+/ud7n7Dne/CjgXuMvMDgeSk/iXikXpmCT9dNwkrQYs\n2prZY8BCd99ZMG8mcAuwn7vvX+3GqGgrIhJNtYq2XwfeWzjD3f8AHAPcWnnzRESkHga8D9/dfzTA\n/OeBf4ytRSIiEgvdltmAovQCVY9RkexQT9sGE6UXqHqMimSLeto2mCi9QNVjVCT9qtrT1szeZWa/\nNLMnctMzzezS4TZS4hGlF6h6jIpkS5gc/neBS4Au6LtT55Q4GyWVi9ILVD1GRbIlTMAf6+6lQyl0\nx9EYGb4ovUDVY1QkW8IE/G25IRUcwMw+BbwYa6ukYlF6garHqEi2DFm0NbODgTXAkcDLwF+Ape5e\nptw3PCraiohEE6VoO+htmWY2Amhz92PNbBwwwt1frUYjRUSktgZN6bh7L8Ggabj7LgV7EZH0CpPD\n/4WZXWRmbzez/fKv2FsmIiJVFaan7X/N/TynYJ4DB1e/OSIiEpchA767v6MWDRERkXgNGfDN7LRy\n8939h9VvjoiIxCVMDn9uwWsBwWMPTxzqQ2bWbGa/M7PHzOyPZvaVYbVU6k4ja4qkW5iUznmF02Y2\nEfjXEOt+A/iAu79mZqMInom73t0fqqypUk8aWVMk/SoZD78TOGSohTzwWm5yVO6VnKE5JZIVK/qD\nfV5nZzBfRNIhTA7/5/QH6hHAdOCnYVZuZk3AJuAfgP/p7r8ts8wyYBnAVI3alVgaWVMk/cLclnlV\nwe/dwGZ33xJm5e7eA7Sa2T7AbWY2w92fKFlmDcHQDbS1tekbQEJNnVp+7Hz9jRZJjzApnQ+5+69z\nrwfcfYuZfT3KRtx9B3AfcHwljZT608iaIukXJuAfV2beCUN9yMwm567sMbMxwLHA09GaJ0mhkTVF\n0m/AlI6ZnQWcDRxsZn8oeGsC8ECIdR8I/CCXxx8B/MTd7xhOY6W+li5VgBdJs8Fy+D8C1gP/Alxc\nMP9Vd//7UCvOPRnrsOE1T0REqmXAgO/uO4GdwKkAZvYWoBkYb2bj3V33Z4iIpEiYh5h/1MyeIXjw\nya+B5wiu/EVEJEXCFG3/GTgc+FNuILVjCJfDFxGRBAkT8LvcfTswwsxGuPu/A60xt0tERKosTMer\nHWY2HtgArDOzlwg6YImISIqEucL/GMH4ORcAdwH/AXw0zkaJiEj1hRktc5eZTQMOcfcfmNlYoCn+\npomISDWFuUvnH4GbgRtysw4Cbo+zUSIiUn1hUjrnAEcBrwC4+zPAW+JslIiIVF+YgP+Gu7+ZnzCz\nkWhcexGR1AkT8H9tZl8GxpjZcQRj4f883maJiEi1hQn4FwNbgceBM4E7gUvjbJSIiFTfYKNlTnX3\n5929F/hu7iUiIik12BV+3504ZnZLDdoiIiIxGizgW8HvB8fdEBERiddgAd8H+F1ERFJosJ62s8zs\nFYIr/TG538lNu7vvHXvrRESkagZ7AIqGTxARaSBhbssUEZEGoIAvIpIRCvgiIhmhgC8ikhEK+CIi\nGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmhgC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmh\ngC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhkRW8A3s7eb2b+b2VNm9kcza49rWyIiMrSR\nMa67G1ju7g+b2QRgk5n9wt2fjHGbIiIygNiu8N39RXd/OPf7q8BTwEFxbU9ERAZXkxy+mbUAhwG/\nLfPeMjPrMLOOrVu31qI56eQ++HRW2iAiFYs94JvZeOAW4AJ3f6X0fXdf4+5t7t42efLkuJuTTqtW\nwYUX9gdY92B61apstUFEhiXWgG9mowiC/Tp3vzXObTUsd9ixA665pj/gXnhhML1jR22uspPQBhEZ\nNvOY/rOamQE/AP7u7heE+UxbW5t3dHTE0p5UKwywee3tsHo1mGWnDSKyBzPb5O5toZaNMeAfDWwA\nHgd6c7O/7O53DvQZBfxBuMOIgi9kvb21D7RJaIOIFIkS8OO8S+d+dzd3n+nurbnXgMFeBpG/ui5U\nmE/PShtEZFjU0zbpClMp7e3BVXV7e3E+PQttEJFhi7PjlVSDGeyzT3G+fPXq4L199qlNSiUJbRCR\nYYsth18J5fAH4V4cWEuns9IGESmSiBy+VFlpYK1HoE1CG0SkYgr4El5Setr29g4+XQtJORYiESjg\nSzhJ6Wm7eDHMmdMf5Ht7g+nFi2vXhqQcC5GIFPBlaEnpadvbCzt3wqOP9gf9OXOC6Z07a3Oln5Rj\nIVIBFW0lnKT0tC0M8nmtrbBpU3GnsDgl5ViIkJCetpVQwE+4pPS07e2Fpqb+6Z6e2gX7vKQcC8k8\n3aWTFlEKf3EVKnt6Bp/Oc4cLSoZEuuCC2qcw8lf4hQpz+rWgXseSUgr49RKl8BdXobKlBQ44oD/I\n9/QE0y0txcu5wxFHwLe+BeefH2z//POD6SOOqG0OP5/OaW0N2tvaWpzTj5t6HUuKKeDXQ5TCX1yF\nyp4e2LULtm3rD/oHHBBM79o18JV+PY0YARMnFufsN20KpidOrE1aZ6Bex+3t6nUsyefuiXnNmTPH\nM6O317293T0I78GrvT2YX6qnx721tXjZ1tZg/nB0d7tPmlS83kmTgvnl2nv++cXLnn9++fbGrXS/\nh3scKlG63/U4DiLuDnR4yBirom09RSn8xVWo7OmBkQVDKnV3F2+n0vaKSE2oaFtPYQuxUQp/UQuV\nYQu8+TROocKcfml7oxRtoxaZk1DAjkI9bSWFFPCrKWwhNkrhL2qhMmyBt6cH9t47yNlPmhRc2U+a\nFEzvvXdx0HcPcuTlirYTJ+4Z7KIWmZNQwI5CPW0lpRTwqyVKITZK4S9KoTJKgXfEiP7UzSmnBNOn\nnBJMNzXtud7u7uD3deuC6XXrgunu7uL1Ri0yJ6GAHYV62kqahU321+KV+qJtlEJsfvnBpguFLVRG\nKfD29rqfe27xsueeW74d3d3u++9fvOz++5cv8EYtMiehgB1F1PMsEiNUtB2Cxziue5QiaBS9vXte\ndQ9UtO3uhlGj+qe7uorbVGl7u7pg9Oj+6TffLN5OpeuF+ArYcZ1rFbAlIVS0HUyc+ddFi8oXQRct\nGt56o+StFy2CCROK502YUL4NUdq7YAGMGVM8b8yYYH6pyy8vv97LL99zWYivgB3XuY7SXpEkCftV\noBav2FM6hV/F81/BS6crVXhPe/5e9tLpShSmMPKpi9LpvK4u9+bm4L3m5vLTlbT3zTfdm5qC95qa\nyk9XehyinJMoxyKucx3nvyGRChAhpVP3IF/4qkkOP87862WXle/IdNllw1tvlLz1ggX9QT7/am4O\n5g+nvVOn9gf5/KupKZhfauHC8utduLD8/q1cWXwO8udo5co9l120qHjf88dm0aI9l43rXEdpr0jM\nogT87Obw48q/xpnDD5u3TmsOP2yuPUo9I65zHaW9IjFSDn8w7sHtj4Xa2wfvIDXYdKHeXmgrOe5t\nbQPfLhh23VHy1r29MHdu8by5cwdeNmx7e3th3rziefPmlV/WHZYvL563fPngxy7K83LDLuse3wif\ner6vpFHYrwK1eNUkh3/QQcFX+/POC6bPOy+YPuigPb/qR/nqHiW/HGXdUdYbddkDDii/7AEHVL7e\nuHPcYY9bb6/7/PnBdvNj/uTHApo/X7l2aRhESOlk6wrfvb8H6YYNwfSGDcF0T0/xlZ97tA42UTpI\nRVl3lPVGWdasP8WyYEEwnb/jpqmp8s5fcY4mGfWciEixsH8ZavGqSdE2asekqEW/sB2koq47ygiR\nUdqQ/4aTf+W/+Qy3DXGNJhnluCVphE+RmKCi7RCidtyJq8Ab57rT1IaoorQ5jfsnEoGKtoOJUgDN\npwwKVauDTZzrjtKGJDy2MIooxy0JxzjfjsGmRWol7FeBWrxiT+kkpfiYhM47aSxqRjluSTjG7rpn\nX2JHhJTOADdnN6iBio9z5oQvPsLwi49xrruRRTluSTjGXlBkhmD7hcNiu+7dl9rKbg4/SseduDrY\nxLnusNu/4IJgXPu888+Hq69OdiCKctyScIzzQT6v8I+QyDBFyeFnM+BLPxU146djLDHKZtFWhbHo\nklLUbGQ6xpIgjRHw9ci56ApTDUM9ZlEqo2MsCZP+oq0KY5VJQlGz0ekYS8I0Rg5fhbHK1buomQU6\nxhKjbBZtk1IY039uEamhRBRtzex/mdlLZvZEXNvok5TCmGoJIpJgcRZt/zdwfIzrDySlMFZYS9BI\njiKSQLEVbd39N2bWEtf6+ySlMFa43Wuu6a8nqJYgIgkRaw4/F/DvcPcZgyyzDFgGMHXq1DmbN2+u\nbGNJyZ0npZYgIpmQiBx+WO6+xt3b3L1t8uTJla8oCY+cS0otQUSkjLoH/IaRlFqCiMgA0t/xKimS\nUksQERlAbDl8M/s3YDEwCfgbsNLdvzfYZxpi8LSk1BJEJBOi5PDjvEvn1LjWnWhJqCWIiJShHL6I\nSEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGREosbDN7OtQIWD6cRuErCt3o2IkfYv3Rp5/xp5\n32D4+zfN3UONS5OogJ9kZtYRtnNDGmn/0q2R96+R9w1qu39K6YiIZIQCvohIRijgh7em3g2ImfYv\n3Rp5/xp536CG+6ccvohIRujfaNa6AAAFaklEQVQKX0QkIxTwRUQyQgG/DDNrMrNHzOyOMu+dbmZb\nzezR3OuMerSxUmb2nJk9nmv7Hg8fsMC3zOxZM/uDmc2uRzsrFWL/FpvZzoLzd3k92lkJM9vHzG42\ns6fN7CkzO6Lk/bSfu6H2L83n7t0F7X7UzF4xswtKlon9/OmJV+W1A08Bew/w/k3ufm4N21Nt73f3\ngTp6nAAcknvNB67P/UyTwfYPYIO7f6Rmramea4C73P1TZjYaGFvyftrP3VD7Byk9d+7+f4FWCC4o\ngf8H3FayWOznT1f4JcxsCvBh4MZ6t6VOPgb80AMPAfuY2YH1blTWmdnewELgewDu/qa77yhZLLXn\nLuT+NYpjgP9w99JRBWI/fwr4e7oa+G9A7yDLnJT7ynWzmb29Ru2qFgfuMbNNZraszPsHAX8tmN6S\nm5cWQ+0fwBFm9piZrTezQ2vZuGE4GNgKfD+XbrzRzMaVLJPmcxdm/yCd567UKcC/lZkf+/lTwC9g\nZh8BXnL3TYMs9nOgxd1nAvcCP6hJ46rnKHefTfD18RwzW1jyfrlnMqbp3t2h9u9hgrFHZgHfBm6v\ndQMrNBKYDVzv7ocBu4CLS5ZJ87kLs39pPXd9cqmqE4Gflnu7zLyqnj8F/GJHASea2XPAj4EPmNna\nwgXcfbu7v5Gb/C4wp7ZNHB53fyH38yWCHOK8kkW2AIXfWqYAL9SmdcM31P65+yvu/lru9zuBUWY2\nqeYNjW4LsMXdf5ubvpkgQJYuk9ZzN+T+pfjcFToBeNjd/1bmvdjPnwJ+AXe/xN2nuHsLwdeuX7n7\n5wqXKcmpnUhQ3E0FMxtnZhPyvwNLgCdKFvsZcFrujoHDgZ3u/mKNm1qRMPtnZgeYBU+WN7N5BP8H\ntte6rVG5+38CfzWzd+dmHQM8WbJYas9dmP1L67krcSrl0zlQg/Onu3RCMLOvAh3u/jPgfDM7EegG\n/g6cXs+2RfRW4Lbc/5mRwI/c/S4z+ycAd/8OcCfwIeBZoBP4fJ3aWokw+/cp4Cwz6wZeB07x9HQ3\nPw9Yl0sL/Bn4fAOdOxh6/9J87jCzscBxwJkF82p6/jS0gohIRiilIyKSEQr4IiIZoYAvIpIRCvgi\nIhmhgC8ikhEK+NIwzKynZETClgrWsY+ZnV391vWt/z1mttHM3jCzi+Lajkg5ui1TGoaZvebu44e5\njhbgDnefEfFzTe7eE2K5twDTgI8DL7v7VZW0U6QSusKXhmbBsw2+YWa/zw14d2Zu/ngz+6WZPWzB\n+Pkfy33kvwPvzH1D+IYFY7DfUbC+a83s9Nzvz5nZ5WZ2P/BpM3unmd2VG7htg5m9p7Q97v6Su/8e\n6Ip950VKqKetNJIxZvZo7ve/uPsngC8QdFGfa2Z7AQ+Y2T0EoxJ+wt1fyY3H8pCZ/YxgwK4Z7p4f\nu3zxENvc7e5H55b9JfBP7v6Mmc0HrgM+UO2dFKmUAr40ktfzgbrAEmCmmX0qNz2R4AETW4Arc6Np\n9hIMQ/vWCrZ5EwTfGIAjgZ/mhnYA2KuC9YnERgFfGp0B57n73UUzg7TMZGCOu3flRkhtLvP5bopT\nn6XL7Mr9HAHsKPMHRyQxlMOXRnc3wYBbowDM7F25kTQnEjz7oMvM3k9QSAV4FZhQ8PnNwHQz28vM\nJhKM4rgHd38F+IuZfTq3HTOzWfHskkhldIUvje5GoAV4ODe07laCO2TWAT+34EHnjwJPQ/C8AzN7\nwMyeANa7+xfN7CfAH4BngEcG2dZS4HozuxQYRfBMhccKFzCzA4AOgucl91rwIOvpuT8YIrHSbZki\nIhmhlI6ISEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6ISEb8f7vahkhdNuLQAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb486cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Original Data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. PLA算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征归一化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先分别对两个特征进行归一化处理，即："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$X=\\frac{X-\\mu}{\\sigma}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$\\mu$是特征均值，$\\sigma$是特征标准差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 270,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEWCAYAAACaBstRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2cVnWd//HXhwEEBFGBFgNhrMVN\nJJhgALVUdr2vVjPTMLayjcUfeYP+sE2XFH62uPWoX95k3tB9K1uamplJmLWupNgyGCneJGSiE24C\nCSojwsx89o9zrplrLq4Zzpm5znXOdV3v5+NxPWbOzZzzub4D12fO93O+32PujoiISBT90g5AREQq\nh5KGiIhEpqQhIiKRKWmIiEhkShoiIhKZkoaIiESmpCESkZktMbPbwu/HmdkbZlZX4nMca2a/L+Ux\nezjXC2Z2YjnOJdVDSUMyI/wQ+7OZ7Z+3bq6ZPZRiWEW5+4vuPtTd2/pyHDNzM/vrvOOucve/6XuE\npVUYp9QuJQ3Jmv7Agr4exAL69y1SYvpPJVnzZeAyMzuw2EYzO8bM1pjZjvDrMXnbHjKzpWb2CNAC\nvCNc969m9mjYnfRTMxthZsvN7LXwGPV5x7jezF4Kt601s2O7iaM+/Ou7v5kdHR4799plZi+E+80w\ns9Vmtt3MXjazG81sYLjt4fBwvwt/7qNmNsvMmvPOc0T4Hrab2VNmdnretu+a2dfN7Gdm9rqZ/cbM\n3tldw5rZx81sk5ltM7NFBdvixnmQmd1nZlvM7NXw+7HdnVuqh5KGZE0T8BBwWeEGMzsY+BlwAzAC\n+CrwMzMbkbfbx4F5wDBgU7hudrh+DPBOYDXwHeBg4Blgcd7PrwEawm3/AfzIzAb1FLC7rw67qoYC\nBwGPAT8IN7cBlwIjgaOBE4DPhD93XLjPlPDnby94vwOAnwIPAG8DLgKWm1l+99W5wP8Lz7sRWFos\nRjObCNwctsPbCdov/0M+bpz9CNpwPDAOeBO4sad2kuqgpCFZdBVwkZmNKlj/AWCDu/+7u7e6+w+A\nZ4G/z9vnu+7+VLh9T7juO+7+B3ffAawA/uDuD7p7K/Aj4D25H3b329x9W/jz/x/YD4hTY7gB2Aks\nCo+31t0fC4/3AnArcHzEYx0FDAW+6O673f1XwH0EiSLnbnf/7/C9LCdIeMV8BLjP3R9297eAK4H2\n3Ma4cYZtdJe7t7j76wTJKur7kgrWP+0ARAq5+3ozuw+4nOBKIOftdF495GwiuILIeanIIf+c9/2b\nRZaH5hbMbCEwNzyXAwcQ/PW9T2Z2PjALOMrd28N1hxNcETUCQwj+z62NcrwwhpdyxwoVvt//yfu+\nJf+9FDtWbsHdd5rZtrzYY8VpZkOAa4FTCa5yAIaZWV1fbw6QbNOVhmTVYuCf6PoBuZmgOyTfOOBP\necu9nrY5rF98DjgHOMjdDwR2ABbxZ78AnBFe0eTcTHA1NMHdDwD+JcrxQpuBQwsK+oXvN6qXgUPz\n4h1C0EXV2zgXElyBzQz3z3VhRX1vUqGUNCST3H0jcDtwcd7q+4HDzexjYQH6o8BEgi6bUhgGtAJb\ngP5mdhXBlUaPzOzQMNZPuPtzRY75GvCGmb0LmF+w/c/AO7o59G8Iurr+2cwGmNksgq64H0Z7O13c\nCXzQzN4XFrivpuv//7hxDiO4Stse1poWIzVBSUOy7GqgY8yGu28DPkjwV+424J+BD7r71hKdbyVB\nzeM5gm6gXRTv7ip0AjAauDPvDqqnwm2XAR8DXge+QZBc8i0BvhfetXRO/gZ33w2cDpwGbAVuIkhM\nz8Z9Y+7+FHABQXH/ZeBVoDlvl7hxXgcMDuN6DPh53JikMpkewiQiIlHpSkNERCJT0hARkciUNERE\nJDIlDRERiazqBveNHDnS6+vr0w5DRKSirF27dqu7F87CsJeqSxr19fU0NTWlHYaISEUxs8LZFopS\n95SIiESmpCEiIpEpaYiISGRVV9MoZs+ePTQ3N7Nr1660Q6lIgwYNYuzYsQwYMCDtUEQkZTWRNJqb\nmxk2bBj19fWYaRLOONydbdu20dzczGGHHZZ2OCKSsprontq1axcjRoxQwugFM2PEiBG6ShMRoEaS\nBqCE0Qdqu+q3fDnU10O/fsHX5cvTjkiyqia6p0Ske8uXw7x50NISLG/aFCwDzJmTXlySTTVzpZG2\nuro6GhoamDRpEmeffTYtuf+hMcydO5enn34agGuuuabLtmOOOaYkcUrtWbSoM2HktLQE60UKKWmU\nyeDBg1m3bh3r169n4MCB3HLLLbGP8c1vfpOJEycCeyeNRx99tCRxSu158cV466W2pZo0zOzbZvaK\nma3vZvssM9thZuvC11XliCvp/t1jjz2WjRs3AvDVr36VSZMmMWnSJK677joAdu7cyQc+8AGmTJnC\npEmTuP324CFqs2bNoqmpicsvv5w333yThoYG5oT9B0OHDgXgox/9KPfff3/Huc477zzuuusu2tra\n+OxnP8v06dOZPHkyt956a2nflFSscePirZca5+6pvQgeRj8VWN/N9lnAfXGOOW3aNC/09NNP77Wu\nO7fd5j5kiDt0voYMCdb3xf777+/u7nv27PHTTz/db7rpJm9qavJJkyb5G2+84a+//rpPnDjRH3/8\ncb/zzjt97ty5HT+7fft2d3c//vjjfc2aNV2OV3j8u+++2z/xiU+4u/tbb73lY8eO9ZaWFr/11lv9\nC1/4gru779q1y6dNm+bPP/985PjjtKFUlqT+zdeC225zHz/e3Sz4WsltBjR5hM/YVK803P1h4C9p\nxlAoqf7d3JVBY2Mj48aN49Of/jS//vWvOfPMM9l///0ZOnQoH/7wh1m1ahXvfve7efDBB/nc5z7H\nqlWrGD58eOTznHbaafzqV7/irbfeYsWKFRx33HEMHjyYBx54gO9///s0NDQwc+ZMtm3bxoYNG/r2\npqQqzJkDy5bB+PFgFnxdtkxF8H3J3UCwaVOQanM3EFT7nWeVcPfU0Wb2O2AzcJm7P1W4g5nNA+YB\njOvjNXVS/bu5mkY+7+b57Icffjhr167l/vvv54orruDkk0/mqqui9cwNGjSIWbNmsXLlSm6//XbO\nPffcjnN97Wtf45RTTunbG5GqNGeOkkRcPf2BWc1tmfVC+OPAeHefAnwNuKfYTu6+zN0b3b1x1Kh9\nTgffo3L27x533HHcc889tLS0sHPnTn784x9z7LHHsnnzZoYMGcI//MM/cNlll/H444/v9bMDBgxg\nz549RY87e/ZsvvOd77Bq1aqOJHHKKadw8803d/zMc889x86dO0v/pkRqRK3eQJDppOHur7n7G+H3\n9wMDzGxkkudcuhSGDOm6bsiQYH2pTZ06lfPOO48ZM2Ywc+ZM5s6dy3ve8x6efPJJZsyYQUNDA0uX\nLuXzn//8Xj87b948Jk+e3FEIz3fyySfz8MMPc+KJJzJw4EAguF134sSJTJ06lUmTJnH++efT2tpa\n+jclUiNq9gaCKIWPJF9APd0XwkcDFn4/A3gxt9zdq6+FcPfqKm6VigrhkrRK+3+X5A0EabQFEQvh\nqdY0zOwHBHdIjTSzZmAxMADA3W8BPgLMN7NW4E1gdvjmEqX+XZHyqsRR6bm4Fi0KuqTGjQt6JPoa\nb9bbwsrwGVxWjY2NXvi412eeeYYjjjgipYiqg9pQklRfH3w4Fho/Hl54odzRpCuttjCzte7euK/9\nMl3TEJHaUKtF5WKy3hZKGiJVqpJmrq2FonLU30fW20JJQ6QKVdrAs3LetZiGOL+PrLeFkoZIFaq0\nmWurfVR6nN9H1ttCSaNMzIyFCxd2LH/lK19hyZIlJT+PpkwXyH6/eDFz5gSF3vb24GtWPiRLIe7v\nI8ttoaRRTOEdZSW4w2y//fbj7rvvZuvWrX0+Vk80ZbpA9vvFa001/T6UNAotWQKXXtqZKNyD5T5e\nFfTv35958+Zx7bXX7rVty5YtnHXWWUyfPp3p06fzyCOPdKw/6aSTmDp1Kueffz7jx4/vSDof+tCH\nmDZtGkceeSTLli0D0JTp0iHJfvHPfAb69w+6Tvr3D5ZLoZIK9zlRY07y91H2dosyArCSXn0aEd7e\n7r5gQTC0c8GC4su9tP/++/uOHTt8/Pjxvn37dv/yl7/sixcvdnf3c88911etWuXu7ps2bfJ3vetd\n7u5+wQUX+DXXXOPu7itWrHDAt2zZ4u7u27Ztc3f3lpYWP/LII33r1q0d5yk8r3vfp0zXiPDKk8So\n4vnzu46Azr3mz+97rJU2PXvcmJP4fZSy3Yg4Ijz1D/lSv/o8jUh+osi9+pgw3Ds/vK+88kq/+uqr\nuySNUaNG+ZQpUzpeb3/72/21117zKVOmdPkAP+iggzqSxuLFi33y5Mk+efJkP+CAA3z16tVdzlN4\n3jfffNPHjh3ru3bt8nvuucc/9rGPubv7WWed5RMmTOg4d319va9cuXKv+JU0xN29rq540qir69tx\nx48vftzx40sRdTKyEHMpY4iaNCphavTyMoNrr4Xrr+9cd+21wfoSuOSSS5g6dSqf+tSnOta1t7ez\nevVqBg8e3GXf4Pe4t4ceeogHH3yQ1atXM2TIEGbNmsWuXbt6PK+mTJdSaGuLtz6qSizcZyHmNGJQ\nTaOQhzWMfPk1jj46+OCDOeecc/jWt77Vse7kk0/mxhtv7FjOPXfjfe97H3fccQcADzzwAK+++ioA\nO3bs4KCDDmLIkCE8++yzPPbYYx0/qynTJUl1dfHWR1WJheIsxJxGDEoa+XIJ4/rrYcGC4H63BQuC\n5RImjoULF3a5i+qGG26gqamJyZMnM3HiRG655RYAFi9ezAMPPMDUqVNZsWIFhxxyCMOGDePUU0+l\ntbWVyZMnc+WVV3LUUUd1HEtTpkuSchPnRV0fVdID2uIUi7NQ3I4qlRii9GFV0qvPNY3Fi7vWMHI1\njrD+UE67du3yPXv2uLv7o48+6lOmTCl7DDmqaYh7UGDt379r/3n//tmeDjxOsTgLxe24ShUDEWsa\nmuW2GPeuNYzC5TLZsGED55xzDu3t7QwcOJCbbrqJ6dOnlz0O0Cy3EqjE2WjjxFyJ769Uos5yq0J4\nMYUJIoWEATBhwgR++9vfpnJukWKyUPyNK07Mlfj+yq1mahrVdkVVTmo7yclC8TeuODFX4vsrt5pI\nGoMGDWLbtm368OsFd2fbtm0MGjQo7VBKLqmRtEkUXZMWdZR33MJrFto4TsxZKG5nXU10T40dO5bm\n5ma2bNmSdigVadCgQYwdOzbtMEoqqUdqxjluVh7r+ZnPwM03dy63tXUu33RT133jPOI0C20cN+ak\nHuFaTWqiEC5SKKmCZyUWXfv3Lz44r64O+nL3dRbaWKKLWghX0pCa1K9f8WE3ZsHwnHIcN6kY4urp\nPo++fDxkoY0lOj0jXKQHSRU8K7HoWmmjvLPSbrVKSUNqUlIF3UosumZllHdWRmJX4o0MZRVlBGAl\nvYqNCBcpJupI2iRHCWdhRLF7MLV5bgbburq+T3Wek4U2jhtvUqPHsw6NCBcpDRVek5eVNq7EGxlK\nRYVwkRJR4TV5WWnjSryRoVQqohBuZt82s1fMbH03283MbjCzjWb2hJlNLXeMIiq89l7UPv+4bZxU\nLaESb2Qot7QL4d8FTu1h+2nAhPA1D7i5h31FEpGVgnWlyQ3C27Qp+Is8Nwiv2Ad8nDaOc9y4KvFG\nhrKLUvhI8gXUA+u72XYrcG7e8u+BQ3o6ngrhkoSsFKwrSdxHkUZt46Qfs1qJNzKUApVSCDezeuA+\nd59UZNt9wBfd/dfh8i+Bz7l7U8F+8wiuRBg3bty0TcWqUyJSVhrcV1kqoqYRQbGxqnv9c3H3Ze7e\n6O6No0aNKkNYIrIvGtxXnbKeNJqBQ/OWxwKbU4pFJJI4Rdqos8smHUcSkurzr9laQlZE6cNK8kXP\nNY0PACsIrjiOAv57X8dTTUPSFGfA1/z5xfvmSzGwLisDz5IchFcttYSsoBJqGmb2A2AWMBL4M7AY\nGADg7reYmQE3Etxh1QJ8ygvqGYU0TkPSFGfAV1Kzy8aNQwQ0uC/tMKRGxSnSJjW7bNw4RKB6CuEi\nFSVOkTap2WXjxiESh5KGSAktXQoDB3ZdN3Bg8SJt3Nllk3rEqUgcShoiJVbYLdRdV9N73xvUNfL1\n7x+sLxR3FPScObBsWVDDMAu+Llumx5ZK36mmIVJCSc2SqsK2JE01DZEUvPhi9PVJ7SuSJCUNkRJK\napZUFbYlK5Q0RCKIOnI7qVlSly6FAQO6rhswQI84lRREGQFYSS+NCJdSiztyO4lZUm+7zX3gwK7n\nHziwNCOha/kRp9KJShgRngQVwqXUkhy5HVWShXAV5AVUCBcpmWIJo6f1SUiyEK6CvMShpCGZloX+\n87gjt5OIOclCuAryEoeShmRWko/1jCPOyO2kYn7/++Otj0OPOJVYohQ+KumlQnj1SPqxnnHMn+9e\nVxecv66u+yJ4UjHrEaeSNFQIl0pXiTO16hGnUqlUCJeKV4n953rEqVQ7JQ3JrCz1n0ctbusRp1L1\novRhVdJLNY3qkoX+87gD2vSIU6lEqKYhUhoa0Ca1QDUNkRLRgDaRTkoaIvugIrRIJyUNkX1IeoZZ\nkUqipCESgVnPyyK1QklDZB8WLYLdu7uu2707WC9Sa5Q0RPZBhXCRTkoaIvugQrhIp1SThpmdama/\nN7ONZnZ5ke3nmdkWM1sXvuamEafUtrijsbMwnbtIUvqndWIzqwO+DpwENANrzOxed3+6YNfb3f3C\nsgcoEpozJ/i6aFHQJTVuXJAwcuvz5aZGb2kJlnNTo+cfR6SSpXmlMQPY6O7Pu/tu4IfAGSnGI9Kt\nOXOC0d/t7cHX7hLAokWdCSOnpUVFc6keaSaNMcBLecvN4bpCZ5nZE2Z2p5kdWuxAZjbPzJrMrGnL\nli1JxCoSiYrmUu3STBrF7nQvnAjrp0C9u08GHgS+V+xA7r7M3RvdvXHUqFElDlOiUD9+IG7RXO0m\nlSbNpNEM5F85jAU25+/g7tvc/a1w8RvAtDLFJjFk5bGsWRCnaK52k0qUZtJYA0wws8PMbCAwG7g3\nfwczOyRv8XTgmTLGJxGpH7/TnDmwbFkwA65Z8HXZsuI1ELWbVKJUp0Y3s/cD1wF1wLfdfamZXU0w\nr/u9ZvZvBMmiFfgLMN/dn+3pmJoavfz0KNLeUbtJlvR5anQze7eZPWZmL5nZMjM7KG/bf5ciSHe/\n390Pd/d3uvvScN1V7n5v+P0V7n6ku09x97/dV8KQdGjwW++o3aQS9dQ9dTOwBHg38BzwazN7Z7ht\nQHc/JLVHjyLtHbWbVKKeksZQd/+5u293968AFwI/N7Oj2PsuJ6lhcfrxpZPaTSpRtzUNM/sdcJy7\n78hbNxm4CzjY3UeUJ8R4VNMQEYmvFI97/RJwRP4Kd38COAG4u2/hiYhIJep27il3/49u1r8I/FNi\nEYmISGZpanTpVpzRyhrZLFIbUpvlVrItzmytmtlVpHakOrgvCSqEl0Z9ffDhX2j8+GCW197uKyLZ\nVIpCeO5Ah5vZL81sfbg82cw+X4ogJbvizNaqmV1FakeUmsY3gCuAPdBxB9XsJIOS9MUZrayRzSK1\nI0rSGOLuhdOGtCYRjGRHnNHKGtksUjuiJI2t4fQhDmBmHwFeTjQqSV2c0coa2SxSO/ZZCDezdwDL\ngGOAV4E/AnPcvUjpM30qhIuIxBe1EN7jLbdm1g9odPcTzWx/oJ+7v16qIEVEpLL02D3l7u0EExXi\n7juVMEREaluUmsYvzOwyMzvUzA7OvRKPTEREMifKiPB/DL9ekLfOgXeUPhwREcmyfSYNdz+sHIGI\niEj27TNpmNkniq139++XPhwREcmyKDWN6XmvYwkeAXt6gjFJldOMuCKVK0r31EX5y2Y2HPj3xCKS\nqqYZcUUqW2+ep9ECTCh1IFIbFi3qTBg5LS3BehHJvig1jZ8STiFCkGQmAj9KMiipXpoRV6SyRbnl\n9it537cCm9y9OaF4pMqNG1f82RuaEVekMkTpnnq/u/9X+HrE3ZvN7EuJRyZVSTPiilS2KEnjpCLr\nTit1IFIbNCOuSGXrNmmY2XwzexL4GzN7Iu/1R+CJUpzczE41s9+b2UYzu7zI9v3M7PZw+2/MrL4U\n55V0zZkTPAa2vT34qoQhUjl6qmn8B7AC+Dcg/wP9dXf/S19PbGZ1wNcJrmSagTVmdq+7P52326eB\nV939r81sNvAl4KN9PbeIiPROt1ca7r7D3V9w93PDZ2e8SXAX1VAzK0XZcgaw0d2fd/fdwA+BMwr2\nOQP4Xvj9ncAJZmYlOLeIiPTCPmsaZvb3ZraB4OFL/wW8QHAF0ldjgJfylpvDdUX3cfdWYAcwokiM\n88ysycyatmzZUoLQRESkmCiF8H8FjgKeCycvPAF4pATnLnbFUPgYwSj74O7L3L3R3RtHjRpVgtBE\nRKSYKEljj7tvA/qZWT93/0+goQTnbgYOzVseC2zubh8z6w8MB/pcTxERkd6JMrhvu5kNBVYBy83s\nFYJBfn21BphgZocBfwJmAx8r2Ode4JPAauAjwK98Xw81FxGRxES50jiDYL6pS4CfA38A/r6vJw5r\nFBcCK4FngDvc/Skzu9rMcrPofgsYYWYbgf9L17u4RESkzKLMcrvTzMYDE9z9e2Y2BKgrxcnd/X7g\n/oJ1V+V9vws4uxTnEhGRvoty99Q/Edzuemu4agxwT5JBiYhINkXpnroAeC/wGoC7bwDelmRQIiKS\nTVGSxlvh4Dug4y4mFaNFRGpQlKTxX2b2L8BgMzuJ4FkaP002LBERyaIoSeNyYAvwJHA+QeH680kG\nJSIi2dTt3VNmNs7dX3T3duAb4UtERGpYT1caHXdImdldZYhFREQyrqekkT/v0zuSDkRERLKvp6Th\n3XwvIiI1qqcR4VPM7DWCK47B4feEy+7uByQenYiIZEq3ScPdSzJViIiIVI8ot9yKiIgAShoiIhKD\nkoaIiESmpCEiIpEpaYiISGRKGiIiEpmShoiIRKakISIikSlpiIhIZEoaIiISmZKGiIhEpqQhIiKR\nKWmIiEhkShoiIhJZKknDzA42s1+Y2Ybw60Hd7NdmZuvC173ljlNERLpK60rjcuCX7j4B+GW4XMyb\n7t4Qvk4vX3giIlJMWknjDOB74fffAz6UUhwiIhJDWknjr9z9ZYDw69u62W+QmTWZ2WNm1m1iMbN5\n4X5NW7ZsSSJeERGh52eE94mZPQiMLrJpUYzDjHP3zWb2DuBXZvaku/+hcCd3XwYsA2hsbPReBSwi\nIvuUWNJw9xO722ZmfzazQ9z9ZTM7BHilm2NsDr8+b2YPAe8B9koaIiJSHml1T90LfDL8/pPATwp3\nMLODzGy/8PuRwHuBp8sWoYiI7CWtpPFF4CQz2wCcFC5jZo1m9s1wnyOAJjP7HfCfwBfdXUlDRCRF\niXVP9cTdtwEnFFnfBMwNv38UeHeZQxMRkR5oRLiIiESmpCEiIpEpadQa956XayUGEekVJY1asmQJ\nXHpp54e0e7C8ZEltxSAivaakUSvcYft2uP76zg/tSy8NlrdvL89f+1mIQUT6xLzK/qM2NjZ6U1NT\n2mFkU/6HdM6CBXDttWBWOzGIyF7MbK27N+5zPyWNGuMO/fIuMNvby/9hnYUYRKSLqElD3VO1JPdX\nfr78+kKtxCAivaakUSvyu4UWLAj+ul+woGt9oRZiEJE+SWVEuKTADA48sGv94Nprg20HHlie7qEs\nxCAifaKaRq1x7/rhXLhcKzGISBeqaUhxhR/Otfxh3d7e83I5aKCjVBglDSmvrAzumzULpk3rTBTt\n7cHyrFnliyErbSESg5KGlE9WBve1t8OOHbBuXWfimDYtWN6xozxXHFlpC5GYVNOQ8srK4L78RJHT\n0ABr13YdQ5KkrLSFCBrcl3YY0pOsDO5rb4e6us7ltrbyJYycrLSF1DwVwmtJnGJqUsXftrael3Pc\n4ZJLuq675JLyd8fkrjTy5dc4ykEDHaUCKWlUujjF1KSKv/X1MHp0Z6JoawuW6+u77ucORx8NN9wA\nF18cnP/ii4Plo48ub00j1zXV0BDE29DQtcaRNA10lAqlpFHJ4hRTkyr+trXBzp2wdWtn4hg9Olje\nubP7K4409esHw4d3rWGsXRssDx9eni6q7gY6LliggY6SaappVLo4xdSkir/5iSJn5Ej4n//pWjPI\nxXvJJcHVRc7FF8N115X/g7K9fe96Qho1DQ10lAxQIbyWxCmmJlX8bWuD/nmz0rS27p0wehOviJSF\nCuGVLE5hO04xNW7xN2rRPHelkS+/xlEYb5xCeNzCfRZuCohDI8KlwihpZE2cwnacYmrc4m/Uonlb\nGxxwQNA1NXJkcIUxcmSwfMABXROHe1AzKFYIHz587w/MuIX7LNwUEIdGhEsFUtLIkrijhOMUU+MU\nf+MUzfv16+yGmj07WJ49O1iuq9v7uK2twffLlwfLy5cHy62tXY8bt3CfhZsC4tCIcKlU7l5Vr2nT\npnlFa293X7DAPfjYCF4LFgTre/qZnpbztbX1vJy/vqGhaxwNDcX3b293v/DCrvteeGHxOFpb3UeM\n6LrviBHB+r7EkIsjatvFPXYSevO7FkkI0OQRPmNTKYSb2dnAEuAIYIa7F61cm9mpwPVAHfBNd//i\nvo5d1kJ4Une+xCkqxxXnjqHWVhgwoHN5z56uceWLE/OePTBwYOfy7t1dz9Pb40JyNwUk9bvWTQGS\nEVkvhK8HPgw83N0OZlYHfB04DZgInGtmE8sTXgRJ9Ucff3zxovLxx/ftuBCvH//442HYsK7rhg0r\nHkecmI89FgYP7rpu8OBgfaGrrip+3Kuu2ntfSO6mgKR+13HiFcmKKJcjSb2Ah4DGbrYdDazMW74C\nuGJfxyxL91R+t0KuO6FwuTdaW91HjgyOM3Jk8eXeyu+OyXXDFC7n7NnjPmhQsG3QoOLLvYl59273\nurpgW11d8eXetkWc30mctkjqd53UcUV6iYjdU1l+3OsY4KW85WZgZrEdzWweMA9g3LhxyUeW/5jS\n66/vHFjX1xlK6+pg/ny4+ebg7qNct8zIkcH6vnRR5QrfuYJv7ljFBvf17w/Tp8OaNbBrV2fX0aBB\nwfr87qI4MQ8YAGPGwJ/+FHTXFHW/AAAIGklEQVQF5bqo6uqC9fldVHV1MHEiPP303sedOHHvtojz\nKNnubgqYNm3vmwKS+l3r0bdSoRKraZjZg8DoIpsWuftPwn0eAi7zIjWNsO5xirvPDZc/TlD/uKin\n85a9ppFEf3TSNY2o/fiVWtOIWnuIU99J6nedVK1EJKbUaxrufqK7Tyry+knEQzQDh+YtjwU2lz7S\nXnIP/krMt2BB9/3Rheu726+9HRoLfm+NjT3fBhrn2FH78dvbg6uKfNOnd79v1Jjb22HGjK7rZswo\nvq87LFzYdd3ChT33+cd5nG3UfT3mgMQ49PhdqTRR+rCSetFzTaM/8DxwGDAQ+B1w5L6OWbaaxpgx\nQf/zRRcFyxddFCyPGbN3f/TixV37qXP914sXd90vTl97kseOu+/o0cX3HT2698dNus8/aru1t7vP\nnBmc9+KLg+WLLw6WZ85U7UGqBhFrGqncPWVmZ5pZM0Gx+2dmtjJc/3Yzux/A3VuBC4GVwDPAHe7+\nVBrx7sW9c6TzqlXB8qpVwXJbW9e/QD3GIK64s68mdew4+5p1dhcde2ywnLsTqq6u9wMMk5wFNk67\niUhXUTJLJb3KNrgv7uC3OIO4og7AS/rYUffNv9LKvXJXYH2NIc7AxTjitFv+1UXulbvqEKkSZHlw\nX5LKWgiPOzgsqUFcSR67kmKIK07Mlfj+RGJIvRBe9eIUlXPdH/lKNYgryWPHiSELj3CNI067ZaGN\nc3H0tCxSDlEuRyrpVZbuqawUdLMwQKwSC8Vx2i0LbewevXAv0ktUweC+7Io7OCypQVwaINY7cdot\nC23seYV7CM6fPyW+a2yHlI9qGn0Rd3BYUoO4kjx21PNn5RGuccRptyy0cdTH+or0gh73KuWlQnHy\n1MaSIBXCe0vFxviyUiiuZmpjyQgljXx6/GZ8+d0m+3rkrPSO2lgyRIXwHBUbeycLheJqpzaWDFFN\nI5+Kjb2XdqG4FqiNJUEqhPdWFoqN+nAQkTJTIbw3slBsVF1FRDJMSSMnC8XG/LqKZl8VkQxSITwn\nC8XGpB4tKiJSIqppFMpCPSELdRURqSmqafRW2o/fzEJdRUSkG0oaWZKFuoqISA9U08iSLNRVRER6\noJpGFmWhriIiNUU1jUqWdl1FRKQbShoiIhKZkoaIiESmpCEiIpEpaYiISGRKGiIiEpmShoiIRFZ1\n4zTMbAuwqcimkcDWModTKdQ2PVP7dE9t071Ka5vx7j5qXztVXdLojpk1RRm4UovUNj1T+3RPbdO9\nam0bdU+JiEhkShoiIhJZLSWNZWkHkGFqm56pfbqntuleVbZNzdQ0RESk72rpSkNERPpISUNERCKr\nqaRhZl82s2fN7Akz+7GZHZh2TFlhZmeb2VNm1m5mVXebYG+Y2alm9nsz22hml6cdT5aY2bfN7BUz\nW592LFljZoea2X+a2TPh/6kFacdUSjWVNIBfAJPcfTLwHHBFyvFkyXrgw8DDaQeSBWZWB3wdOA2Y\nCJxrZhPTjSpTvgucmnYQGdUKLHT3I4CjgAuq6d9OTSUNd3/A3VvDxceAsWnGkyXu/oy7/z7tODJk\nBrDR3Z93993AD4EzUo4pM9z9YeAvaceRRe7+srs/Hn7/OvAMMCbdqEqnppJGgX8EVqQdhGTWGOCl\nvOVmqug/vpSHmdUD7wF+k24kpdM/7QBKzcweBEYX2bTI3X8S7rOI4BJyeTljS1uUtpEOxZ6xq/vT\nJTIzGwrcBVzi7q+lHU+pVF3ScPcTe9puZp8EPgic4DU2SGVfbSNdNAOH5i2PBTanFItUGDMbQJAw\nlrv73WnHU0o11T1lZqcCnwNOd/eWtOORTFsDTDCzw8xsIDAbuDflmKQCmJkB3wKecfevph1PqdVU\n0gBuBIYBvzCzdWZ2S9oBZYWZnWlmzcDRwM/MbGXaMaUpvGHiQmAlQSHzDnd/Kt2ossPMfgCsBv7G\nzJrN7NNpx5Qh7wU+Dvxd+Dmzzszen3ZQpaJpREREJLJau9IQEZE+UNIQEZHIlDRERCQyJQ0REYlM\nSUNERCJT0hApYGZtebdKrgungoh7jAPN7DOlj67j+O8ys9Vm9paZXZbUeUQK6ZZbkQJm9oa7D+3j\nMeqB+9x9Usyfq3P3tgj7vQ0YD3wIeNXdv9KbOEXi0pWGSARmVhc+j2VN+DyW88P1Q83sl2b2uJk9\naWa5mXC/CLwzvFL5spnNMrP78o53o5mdF37/gpldZWa/Bs42s3ea2c/NbK2ZrTKzdxXG4+6vuPsa\nYE/ib14kT9XNPSVSAoPNbF34/R/d/Uzg08AOd59uZvsBj5jZAwQz4Z7p7q+Z2UjgMTO7F7ic4Nkt\nDQBmNmsf59zl7u8L9/0l8H/cfYOZzQRuAv6u1G9SpDeUNET29mbuwz7PycBkM/tIuDwcmEAwseE1\nZnYc0E4wffpf9eKct0PHzKjHAD8KpjACYL9eHE8kEUoaItEYcJG7d5mTK+xiGgVMc/c9ZvYCMKjI\nz7fStTu4cJ+d4dd+wPYiSUskE1TTEIlmJTA/nPIaMzvczPYnuOJ4JUwYf0tQnAZ4nWByzJxNwEQz\n28/MhgMnFDtJ+NyFP5rZ2eF5zMymJPOWROLTlYZINN8E6oHHw6mvtxDcubQc+KmZNQHrgGcB3H2b\nmT1iZuuBFe7+WTO7A3gC2AD8todzzQFuNrPPAwMIHjX7u/wdzGw00AQcALSb2SXAxGp62I9kk265\nFRGRyNQ9JSIikSlpiIhIZEoaIiISmZKGiIhEpqQhIiKRKWmIiEhkShoiIhLZ/wJkNWdz+rsZ5wAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb5a8860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值\n",
    "u = np.mean(X, axis=0)\n",
    "# 方差\n",
    "v = np.std(X, axis=0)\n",
    "\n",
    "X = (X - u) / v\n",
    "\n",
    "# 作图\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Normalization data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X加上偏置项\n",
    "X = np.hstack((np.ones((X.shape[0],1)), X))\n",
    "# 权重初始化\n",
    "w = np.random.randn(3,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显示初始化直线位置："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XecFfW5x/HPA4vSFCl6cxNkFxUL\nrssqC3ZEiQV7iVhI1KjBLiY2LBESr5hcjRpjw+i1JMTYExMLiBrFGhfFbgALSoyKKBYQBfa5f/z2\nwJYDO2f3zM6cc77v12teuzPMmXnOrJ7nzDO/Yu6OiIhIh6QDEBGRdFBCEBERQAlBRETqKSGIiAig\nhCAiIvWUEEREBFBCEBGRekoIIiICKCGIiEi9sqQDyEWfPn28oqIi6TBERArKjBkzPnH3dVvar6AS\nQkVFBbW1tUmHISJSUMxsbpT9VDISERFACUFEROopIYiICFBgzxCyWbp0KfPmzWPJkiVJh1KQOnfu\nTN++fenUqVPSoYhIwgo+IcybN4+11lqLiooKzCzpcAqKu7NgwQLmzZtH//79kw5HRBJW8CWjJUuW\n0Lt3byWDVjAzevfurbsrEQGKICEASgZtoGtXWiZPhooK6NAh/Jw8OemIJE0KvmQkItFMngxjxsDi\nxWF97tywDjB6dHJxSXoUxR1C0jp27Eh1dTWVlZUcfPDBLM78H5eDY489ltdffx2AiRMnNvq37bbb\nLi9xSmk777yVySBj8eKwXQSUEPKiS5cuzJw5k1dffZU11liD6667Ludj3HDDDQwcOBBonhCefvrp\nvMQppe2993LbXupKsbxWcgkh7j/yjjvuyJw5cwC47LLLqKyspLKykiuuuAKARYsWsddeezFo0CAq\nKyu5/fbbARg+fDi1tbWMGzeOr7/+murqakbX38d3794dgEMOOYQHHnhgxbmOOuoo7r77bpYvX86Z\nZ57JkCFDqKqqYtKkSfl9U1IU+vXLbXspy5TX5s4F95XltaJPCu5eMMvgwYO9qddff73ZtlX54x/d\nu3Z1D3/isHTtGra3Rbdu3dzdfenSpb7vvvv6Nddc47W1tV5ZWelfffWVf/nllz5w4EB/4YUX/K67\n7vJjjz12xWsXLlzo7u477bSTP//8842O1/T499xzjx9xxBHu7v7NN9943759ffHixT5p0iS/8MIL\n3d19yZIlPnjwYH/77bcjx5/LNZTCFdd//8WovLzxdcos5eVJR9Y6QK1H+IwtqTuEuGqomW/0NTU1\n9OvXj2OOOYYnn3ySAw44gG7dutG9e3cOPPBApk+fzhZbbMG0adM4++yzmT59Oj169Ih8npEjR/Lo\no4/yzTff8OCDDzJs2DC6dOnC1KlTufXWW6murmbrrbdmwYIFzJ49u21vSorO6NFw/fVQXg5m4ef1\n17fvA+VCKcPEWV5L8zUoqVZGcf2RM88QGgpJubmNN96YGTNm8MADD3DOOeew2267ccEFF0Q6T+fO\nnRk+fDhTpkzh9ttv57DDDltxrt/97nfsvvvubXsjUvRGj06uRVEhtXLq1y/El217W6T9GpTUHUJ7\n1lCHDRvGX/7yFxYvXsyiRYu499572XHHHfnggw/o2rUrP/zhDznjjDN44YUXmr22U6dOLF26NOtx\nDz30UG666SamT5++IgHsvvvuXHvttSteM2vWLBYtWpT/NyXSBoXUyumii6Br18bbunYN29si7deg\npBJCXH/kbLbaaiuOOuoohg4dytZbb82xxx7LlltuySuvvMLQoUOprq7moosu4vzzz2/22jFjxlBV\nVbXioXJDu+22G0888QTf//73WWONNYDQZHXgwIFstdVWVFZWctxxx7Fs2bL8vylJpTSXIBoqpFZO\nuZbXov4NUn8NojxoSMvS1ofK7uEBWnm5u1n4qQdqeqhcyArpQXGxPajNyOVvkNQ1QA+Vsxs9Gt59\nF+rqws801O1EWivtJYiG2vMOvT3l8jdI+zUouYQgUkziKkGceCKUlYVySVlZWG+r0aPhyCOhY8ew\n3rFjWE/rl7I4ykBxlaLyJsptRFqWfJSMpDldw8IVRwnihBOyH/OEE9oWayGVt9JQBsrn9SJiySjx\nD/lcFiWEeOgaFq44PmQ7dsz+AdexY9tiLaRnCLnEGleiy+f1ipoQVDISKWBxdDZbvjy37VGlpbNX\nlH3jLANFlUSLpJLqmCZSjPLd2cwsfBfNtr0t0tDZK+q+ucYaR4e/uK7X6ugOIQ/MjNNPP33F+qWX\nXsqECRPyfh4Niy3toVu33LZHlYbOXlH3TUNroCRiKL2E0PSrT7avQjlac801ueeee/jkk0/afKzV\n0bDY0h5W1cm9rZ3f01BaibpvGsZ9SiKG0koIEybAT3+6Mgm4h/U2fpsvKytjzJgxXH755c3+bf78\n+Rx00EEMGTKEIUOG8NRTT63Yvuuuu7LVVltx3HHHUV5eviKh7L///gwePJjNN9+c66+/HkDDYudB\nHE348l27jluU5qS5DvGSy/vKpR9Q1OPmEm8u+6ahz1K7xxDlyXNalja1Mqqrcx87NjymHzs2+3or\ndevWzT///HMvLy/3hQsX+iWXXOLjx493d/fDDjvMp0+f7u7uc+fO9U033dTd3U866SSfOHGiu7s/\n+OCDDvj8+fPd3X3BggXu7r548WLffPPN/ZNPPllxnqbndW/7sNil0MoojpYguRwzDU0uozYnTcP7\niiuGNPwdkoCanWbRMAlkljYmA/eVH8w///nP/Ze//GWjhLDuuuv6oEGDVizf/e53/YsvvvBBgwY1\n+nDu2bPnioQwfvx4r6qq8qqqKl977bX9mWeeaXSepuf9+uuvvW/fvr5kyRL/y1/+4ocffri7ux90\n0EE+YMCAFeeuqKjwKVOmNIu/FBJCHE0eczlmGppc5tKcNOoQL3G9r1yPm8uQNKU4fI0SwqrU1TX+\nL6yNycB95QfzggULvLy83CdMmLAiIfTu3dsXL17c7DVVVVVZE8Jjjz3m22+/vS9atMjdw8Q5jz32\nWKPzND2vu/sPf/hD/+tf/+qHHXaY33fffe7ufuCBB/pDDz3UYvylkBDMsn/AmGXfP8qHRi7HzPX8\ncch2/szSWnFc19YcNxdRYyimxBE1IZTWMwSvf2bQUMNnCm3Uq1cvRo0axY033rhi22677cZVV121\nYj0zb8IOO+zAHXfcAcDUqVP57LPPAPj888/p2bMnXbt25c033+TZZ59d8VoNi916udSOo06fGFft\nOi6ZISOibo8ijuua63FzETUGTaGZwAL8H/Ax8GqU/dP8DCHjww8/9C5duqy4Q5g/f76PGjXKt9hi\nC99ss838uOOOc3f3jz76yHfZZRffcsst/bTTTvP//u//9iVLlviSJUt8jz328C222MJ/8IMfNLpD\nOOuss3zTTTddURJqeN5vv/3We/Xq5UcdddSKbcuXL/dzzjnHKysrffPNN/fhw4evmLKzoVK4Q4hj\nKIJCq13HMSRFXEM8JN37Nw0lvnyiEEpGwDBgq3ZJCO7u48c3/vDPJIX6D+/2tGTJEl+6dKm7uz/9\n9NM+aNCgdo8hoxQSgns85YpCq12fcMLKZwkdO7Z9fKJcjplrGSiOWKPGkIYSXz4VREIIcVLRbgnB\nvfmdQB6eIbTGrFmzvLq62quqqrympsb/+c9/JhKHe+kkhKiK7dthnHSHUBiUECQyXcPG0lDeKRRx\nfcgnPYJosf03UDQJARgD1AK1/fr1a/ZGX3/9da9L6Ft+Mairq1NCyCKOlihxlEByjSHf1MqoMBRN\nQmi4ZLtDePvtt33+/PlKCq1QV1fn8+fPz9pZTVqWy7fIYp1jIC39EGT1oiYEC/smx8wqgL+7e2VL\n+9bU1HhtbW2jbUuXLmXevHksWbIkngCLXOfOnenbty+dOnVKOpSCU1GRfTTK8vIwzEBDZWXZh4/u\n2BGWLWufGOLQdPRQCAOwtXXMnbiOW6rMbIa717S4Y5SsEdcC3Ab8B1gKzAOOWd3+2e4QRJKSS1kj\n236ZJZs0lFaiiqu0Ukwlm6RRKHcIuch2hyCSlD59YMGC5tt794amA9926BA+qpsyCwOXNZTLt+Ok\n7xCkMES9QyitnsoiCclljoFcxvdPw7j9UjyUEESyiDJM9KefZn9ttu25zDEQ5/SNUYeUTsNQ3ZKA\nKHWltCx6hiDtIWqLoLhGO+3dO/u+vXu37X2Vaht8if4MQXcIIk3Uz0nU4vZcyjVpKO1ELUXlUrKS\n4qKEIKmTdLkiW/PQbNtHj4Yjj1w5WmjHjmE9W7kml9JOLqWoXEQtReVSspLiooQgqZKGYYejDhM9\neTLccsvKRLF8eVhfVaxRp0Ps1Su37VFFHVI6DUN1SzKUECRV0lCuGDMm2vY0xJqLqGWrNJS3JBlK\nCJIqaShXXHMNjBjReNuIEWF7lJjaGmtcJaOoZatcWy5J8VBCkFRJQ7li8mR45pnG2555pm0zpuUi\nzmsQtWwVdT8pLkoIkippKFdELQXFFWsaroGUJiUESZU0lCuiloLiijUN10BKk8YyEmlC4wNJsdFY\nRiKttOeeuW0XKRZKCCJNPPBAbttFioUSgkgTuTQnTbpXtUg+KSGINBG12WcaelWL5JMSgkgTUZt9\nFlpPZZGWKCFIqxVruSRqs89ceyoX6/WS4lGWdABSmJpO85gpl0BxtJcfPbrl99GvX/bmqdlKTsV+\nvaQ46A5BWkXlktx6FOt6SSFQQpBWScMgdEnLpUexrpcUApWMpFVyKZcUsyilJdD1ksKgOwRpFQ3A\nlhtdLykESgjSTJTWMLlMHykasE4Kgwa3k0aatoaB8E226YdX1P1EJHlRB7dTQpBGoo70qRFBRQqH\nRjuVVonaGkatZkSKjxKCNBJ1HJ80THUpIvmlhCCNRG0No1YzIsVHCUEaidoaRq1mRIqPHiqLiBQ5\nPVQWEZGcKCFIu9DQzyLpp7GMJHYa+lmkMJTGHcLXX8Py5UlHUbI09LNIYSiNhDBxImy+Odx0E3z7\nbdLRlBx1YhMpDIkmBDPbw8z+ZWZzzGxcbCeqqYHOneHoo2GjjeDKK5t/ZZXYqBObSGFILCGYWUfg\namAkMBA4zMwGxnKy/faDF1+EBx4IDebHjg0/L7oIFi6M5ZSykjqxiRSGJO8QhgJz3P1td/8W+DOw\nX2xnM4ORI2H69LAMHQrnnx++po4bBx9+GNupS506sYkUhiQTwveA9xusz6vfFr8ddoD77w93DXvu\nCZdcEtpCnngivPNOu4RQakaPDqOg1tWFn0oGIumTZEKwLNuadZs2szFmVmtmtfPnz89vBNXV8Oc/\nw5tvwo9+BDfcAAMGhN9fey2/5xIRSbkkE8I8YP0G632BD5ru5O7Xu3uNu9esu+668UQyYAD8/vfw\n9ttw6qlwzz1QWQn77w/PPRfPOUVEUibJhPA8MMDM+pvZGsChwH0JxgN9+8Jll4X2kOPHwxNPwDbb\nwIgRMG0aFNC4TyIiuUosIbj7MuBkYArwBnCHu6ejTtO7N0yYELrUXnopvPEG7LorbL013HtvKISL\niBSZRPshuPsD7r6xu2/o7ulrhLjWWnD66eFB86RJsGABHHhgKCfdeissXZp0hCIieVMaPZXbas01\nw+A7//oX/OlPUFYGRx4Znj1cfXUYGkNEpMCtMiGY2RZm9qyZvW9m15tZzwb/9s/2CS9lysrgsMPg\npZfgb3+D730PTj45NFm9+GL4/POkIxQRabXV3SFcC0wAtgBmAU+a2Yb1/9Yp5rjSzQz23huefBIe\nfxy23BLOPTd0cjv3XPj446QjFBHJ2eoSQnd3f8jdF7r7pYQHwA+Z2TZk6S9Qksxg2DB46CGYMQN2\n3x1+9avQFfeUU8JDaRGRArG6hGBm1iOz4u6PAQcBfwDK4w6s4Gy1FdxxR2iRdPjhcN11YSC9I48M\n20REUm51CeHXwGYNN7j7y8AI4J44gypom2wCN94YOrmddBLceWcYevvAA+H555OOTkRklVaZENz9\nT+7+bJbt77n7T+INqwisvz5ccUUoG513Hjz2WBhQb9ddw+/q5CYiKaNmp3Fbd1248MKQGP73f+GV\nV2CXXWDbbeGvf1UnNxFJDSWE9rL22nDmmWGoz2uugY8+CmMlVVXBH/8Iy5YlHaGIlDglhPbWuTOc\ncALMnh0SAYTRVTfeGK69FpYsSTY+ESlZLSYEM9vYzB4xs1fr16vM7Pz4QytyZWVhUoCXXw6lo/XW\nC/MxVFSE0tIXXyQdoYiUmCh3CL8HzgGWwoqWRofGGVRJ6dAB9t0XnnkGHn00lJDOPjv0ZTj/fMj3\nHBAiIqsQJSF0dfemQ1Wo4J1vZrDzzjB1amieussuMHHiyjmg33+/5WOIiLRBlITwSf2QFQ5gZj8A\n/hNrVKWupgbuvjvM2jZqVHgIvcEGcPTRYYA9EZEYREkIJwGTgE3N7N/AacDxsUYlwWabwc03w5w5\n4UH0bbeFbQcfDC+8kHR0IlJkVpsQzKwDUOPu3wfWBTZ19x3cXYP0tKfycrjyytCX4Zxz4OGHYfBg\n2GOPMLieOrmJSB6sNiG4ex1hUDvcfZG7f9kuUUl2660HF10UEsPFF8OLL8Lw4bD99vD3vysxiEib\nRCkZPWxmZ5jZ+mbWK7PEHpmsWo8eMG5c6OR21VXwwQewzz4waFAoK6mTm4i0QpSEcDThOcITwIz6\npTbOoCSiLl3CAHqzZ8Mtt4REcPjhYYC9SZPUyU1EctJiQnD3/lmWDdojOImoUyc44gh49VW4917o\n3RuOPz60TLr0UvhSlT4RaZl5C3VnMzsi23Z3vzWWiFajpqbGa2t1c9Ii99DJbeLE8LNnzzBhz6mn\nhmQhIiXFzGa4e01L+0UpGQ1psOxImFZz3zZFJ/EygxEj4JFH4LnnYKed4Je/DFN8/vSnMG9e+z+A\nbno+PQAXSZ2ylnZw91MartfPovaH2CKS/Bo6NDxs7t49JIrf/Q6uvjoMprfzzmE9bhMmwMKFcPnl\nIQb3kJjWWSf8m4ikQmtGO10MDMh3IBIT9/Bh/Mc/Qq9e4QH0ZpuFXtBXXw2HHAIzZ8Z//t/+NiSB\nTDL47W/Ddt0piKRGlGcIf6N+2ApCAhkI3OnuZ8ccWzN6htBKDT+EM449NiSIa68ND51Hjgyd3nbc\nsX3OP3bsyjsGEYlV1GcIURLCTg1WlwFz3X1eG+NrFSWENnAPI6tm1NWFD+OFC8OdwhVXwCefwA47\nhMQwcmR+P6xXdX4RiV0+Hyrv6e6P1y9Pufs8M/t1HmKU9pL5ht5Qpnyzzjphzue5c1cOj7HXXrDl\nlnD77bB8ebznF5HUiJIQds2ybWS+A5GYNCzXjB0bvpmPHdu4pg/QtWtomjpnDtx0U+jUduihsOmm\ncMMN8M038Z6/PTSdv7q957NWSytJuVUmBDM7wcxeATYxs5cbLO8AL7dfiNImZuEuoGHN/vLLw/o6\n6zQv26yxBhx1VHjofNddYZiMn/wkdHK77DL46qvczz9zJlRXh9ebhZ/V1WF7e5WNhg8PAwJmkkBd\nXVgfPrx9zj9hQuMEmEmUamUlaeLuWRegB1AB3AaUN1h6reo1cS+DBw92aaW6utWvr+51U6a4Dx/u\nDu69erlPmOC+YEH0148dG147dmz29bgtX+5eXR3OWV2dfT1OabgGUtKAWo/wGRv5wxhYD+iXWaK+\nLp+LEkITuXzIN/3Qa82H4NNPu++zT/jPpls395/+1P3f/17578uWrTrOU08Nr8ssp57avh+EDZNA\nZmmPZJDRMAlkFiUDaSdRE0KLzxDMbB8zmw28AzwOvAs8mOcbFclVLiWIfJVLtt0W7rsPXq6vGF5+\nOfTvD8cdB7NmwXe+AxUVzV/3i19kP96qtsehQweYMaPxthkzGrd8ilOmVNeQmt1KykT5v+F/gG2A\nWe7eHxgBPBVrVLJ6nkNnr7o6+PzzUK/PJIXBg8P655+37sHqwIFhpFUIH6i33BJGWP3kk3DMhi2T\n3OGzz0ILpoauvDJsb68Hq5n33VDDJBm3zN+oIbW0krRp6RaC+lsN4CWgQ/3v/4xy+5HvpSRKRlHL\nQLmUIHItl0QpLy1b5t6nT+NjmoWfe+3l/tRTK+NsWi5qqWyUS3kryvXSMwQpcUQsGUXpmDYN2B/4\nFdAb+BgY4u7bxZemsiv6jmm5jvnjOXT2qquDjh1Xri9fnr1cMnx4+JafKadkvln36AH/+EfjWBcs\nCBP0ZBx7bGi2+sor4d+GDYNzz4VRo0Kz1YZNV9dcEzp3Du+3NefP9Xqts044/5dfQllZmDtirbVC\nHE1jiEMu70skz/LZMW0/wvhFpwEPAW8B+7QtPGkmlzJQZv+oJYio5ZKo5SX35skAQn+Fysowk9vl\nl8Nbb4V5nxctat6P4Ztvwgdzw/JSLuWtXMtm/fuHvhVDhoT1IUPCev/+8ZeN3Fc2s/3Zz8L6z362\nsjmuykaSFlFuIwjNTb9f/3tXYK0or1vN8Q4GXgPqgJqor0tVyai1zThbOuYppzQuq5xySvZzRS1B\n5FouWb7cfdCgxjEMGtR4v2zloszSp8/K1kbffON+ww3u66wTvWSUS3krzrJZvv++amUkCSKPJaOf\nAGMI/Q82NLMBwHXuPqK1ScjMNqtPBpOAM9w9Uh0oNSWjuIZzzpQVGo4+Wl3dvuWS4cPDtpdeWrlt\n0KBwjIYxdO4cjtPwG37HjuH4TafuLC+HTz9t3Kmta9cwWc977zV/X59+2nhY7lNOCQPxtVfZLK6/\nby6xiuRRPktGJwHbA18AuPtsQp+EVnP3N9z9X205RmJyLe1E1bBc0tCqWgNNmNC42WKmWWPTD6xc\nyiV1dc2TAYT1hQsbN1vdbLPm4xwtXx62Nz1mjx7NezgvXgwffBAm7vnss7DNvXkygLD+6aftUzaL\n6++bS6wiSWnpFgJ4rv7ni/U/y4CXo9x+RDj2PyjEklEct//ZykWrKxvlImq5JFu5KFvZKJdYV1de\n6tQp/Oze3f2ss0Int6ZxrireOMtm+f77qpWRJIx8dUwDHjezc4EuZrYrcCfwt5ZeZGbTzOzVLMt+\nuSQsMxtjZrVmVjt//vxcXhofM/jNbxpv+81vVn37n+2bbbZj9uoVyiMNZcol2Y4d5bgQvVNWhw6h\nLDJoUOPtmZJRZv9MrCef3Hi/k09uHmvHjtCtG/Tp03jfPn3gu98Nd0B77w2XXhrGS/roo3CX0VCm\nbNYw3lzGaOrQIby+unrl+54xI/txM8e+7LLG2zLjMLVGruNJiSSlpYxBKCv9hJAI7qr/3aJkmwjH\n/geFeIcwbFjzb719+oTtTY0f3/hbYObb4fjxzffN5cFnXMeN8lDZ3f3nP89+DX7+8+zH3GKLxvtu\nsUXjY86e7f6Tn7ivscbK/gxR7pCiPvwdP77xcTJ3Odmu1wUXZL9eF1yQ/dhRxdEQQSQC2jqWEe0w\nXlFBJoSGJZBMi5qm6xlxlTXiOm7UfXO5Brm+r2OO8UblpA02yP6+chFneUmkAOQjIbzQ4Pe7oxws\n6gIcAMwDvgE+AqZEeV0qEoJ7bt+Oc6lH77RT4w+dzIfRTju133Gj7pvLXVIu58/c+cyfH0ZV7dkz\nHLuiwn3q1PwkhZauV1x3CLnGu7p1kRxETQirbHZqZi+6+5ZNf09SapqdQmhRU1a2cn3ZssZNGhty\nz61pZNN9VzUAW1zHjbJvtp7KJ58cmpJma5qZ6/vKvI+vvoJJk0IN/4MPQsugc8+F/ffPfWC6XK9X\nlCaqcYir2auUrHw0O/VV/C51dVDT5NrW1GTv8Zr5n7mh1TU3zPagN5u4jhtlX/fQVLRpT+Wrrlr1\ngHW5nL/hh3T37nD66fD223D99eGD8qCDYPPN4eabYenSVR+nacxRr5d76EncUKaHcdzc42n2KhLF\nqm4dgOWEvgdfAsvqf8+sfxHl9iPfS7uUjFq6VY+r1p9rjEk2Y2zNgHX5snSp+223uVdVhfP16+d+\n5ZXuixatPt6o1yvpa9s0hnw0e5WSR1ubnbp7R3df293Xcvey+t8z62vHn6oSEGWOgVyaMMbV3DDp\nZoxm0LMnnHpq4+2nnhq2x3n+srIw1/PMmXD//bD++uG8FRUwcWL2gepyuV5JX9tMDJo7QZIQJWuk\nZYn1DiHXb4b5HqK5tTHHcdyo5056FrSMJ55wHzkyxLD22u7jxrl/+GH2mFe33tp98013CJJn5HsK\nzTQssZeM0vA/YiG0LklDWSWbF15wHzUq9GPo3Nn9pJPc33knmVhaK63XVgpa1ITQTs0mCkTSt+q5\nTIuZpDSUVbLZcku4/XZ4800YPTo8hN5oIzjiCHj99WRiylVar62UhihZIy1LUd8hFOI3w7Tfzbz/\nvvtpp7l37Rqu4/77uz/3XNJRRZP2aysFBZWMcpSGD+Q0lKyK0fz5odNgZl6GESPcp03TdZWSETUh\ntDgfQprE3jEtDR2CPIfOU5KbL78Mndx+8xv48EMYOhTOOQf23bf9Op2JJCBqxzQlhKbcG38AN12P\n+9yZTkgZDWvJkh9LlsAtt8Cvfw3vvAMDB8K4caE5a6dOSUcnknf5nCCntDT94E0iGYwdG+4Mxo5t\n3GNV8qNzZzjuOJg1CyZPDncHRxwBAwbA1VfD118nHaFIIpQQ0kKtS9pfWRkcfniYEe6++8L8DCef\nHDq5/frX8MUXSUco0q5UMkqbJEtWpc4dnngi9HieOjX0PD/ppJCU12vTrLEiiVLJqFAlVbKScK13\n2gmmTIHaWvj+9+Hii8Mdw6mnwnvvJR2hSKyUEESyGTwY7rordGg75BC49lrYcEP48Y9DxzeRIqSE\nILI6m24KN90Eb70FJ54YekIPHAg/+EHzeapFCpwSgkgU/fqFFl9z54YJeqZNC3Ng7LYb/OMfagUm\nRUEJQSQX664L//M/4XnCr34FL78MO+8M220XWiplmyRJpEAoIYi0xtprw9lnh45tV18dej7vtx8M\nGhT6NixblnSEIjlTQhBpiy5dwrOFWbPgD38Idwg//CFsvDFcd13oFS1SIJQQRPKhU6eQCF55Bf7y\nl1BaOuEE6N8fLrkkjKMkknJKCCL51KFDKB09+yw88ghUVsJZZ4WH0hdcAJ98knSEIqukhCASBzPY\nZRd4+GH45z/Dg+cLL4TycjiP4/bPAAAMtElEQVTtNHj//aQjFGlGCUEkbkOGwD33wGuvhf4LV10V\nOrkdc0x49iCSEkoIIu1l4MAw7PacOTBmDPzpT6Hj26hR8OKLSUcnooQg0u4qKsJdwrvvhnkYpkyB\nrbaCkSPD4Hrq5CYJUUIQScp//VcYWfW998LPGTPC4Ho77AD336/EIO1OCUEkaT16hKk8330Xfvc7\nmDcP9t4bqqvhttvUyU3ajRKCSFp07Rom6JkzB26+Gb79Nkzgs+mm8PvfwzffJB2hFDklBJG06dQJ\njjwytEq65x7o2TM8hN5gA/jNb+Crr5KOUIqUEoJIWnXoAAccEPoxPPwwbLIJnHFG6MswYQIsWJB0\nhFJklBBE0s4szN726KPwzDPhofMvfhESw+mnw7//nXSEUiSUEEQKyTbbwF//GsZMOuCAMEfDBhuE\nktKcOUlHJwVOCUGkEFVWhtFVZ88OPZ5vvTWUlA49FF56KenopEApIYgUsv794ZprQpPVM86ABx4I\nzVX32guefDLp6KTAKCGIFIPvfAd+/eswxeeFF4YH0TvuCMOGwYMPqpObRJJIQjCzS8zsTTN72czu\nNbN1kohDpOj07Annnx/uGH772zCj2557hqEx7rgDli9POkJJsaTuEB4GKt29CpgFnJNQHCLFqVs3\nOPVUeOst+L//g6+/hkMOgc02gxtvDJ3eRJpIJCG4+1R3z/THfxbom0QcIkVvjTXgxz8OndzuvBPW\nWguOPTa0TLriCli0KOkIJUXS8AzhaODBpIMQKWodO4a5GGprw+iqG20EP/1p6Mtw4YXw2WdJRygp\nEFtCMLNpZvZqlmW/BvucBywDJq/mOGPMrNbMaufPnx9XuCKlwQx22w3+8Q946inYdtswtWe/fnDm\nmfCf/yQdoSTIPKHWB2Z2JHA8MMLdF0d5TU1NjdfW1sYbmEipefll+NWv4PbboawslJjOOiuUlaQo\nmNkMd69pab+kWhntAZwN7Bs1GYhITKqqwuxts2aFZHDTTTBgAIweHXpES8lI6hnCVcBawMNmNtPM\nrksoDhHJ2HBDuO660FT1Zz8LQ2RUVcE++4QxlKToJdXKaCN3X9/dq+uX45OIQ0Sy+O534ZJLwkxu\nv/gFPP00bLcdDB8eHkirk1vRSkMrIxFJo169wgPnuXPhssvC4Hl77AE1NXD33VBXl3SEkmdKCCKy\net27hyaqb70FN9wAX3wRmrAOHBieN6iTW9FQQhCRaNZcM4ys+uaboUVS585w9NGhT8OVV8JitQ8p\ndEoIIpKbjh1h1Ch48cUwump5OYwdG35edBEsXJh0hNJKSggi0jpmMHIkTJ8elqFDw8B6/frBuHHw\n4YdJRyg5UkIQkbbbYQe4//5w17DnnqGVUkUFnHhiaMYqBUEJQUTyp7oa/vzn8JzhRz8KD6EHDAi/\nv/Za0tFJC5QQRCT/BgyA3/8e3n47DMN9zz1h2s/994fnnks6OlkFJQQRiU/fvqEPw3vvwfjx8MQT\nsM02MGIETJumTm4po4QgIvHr3RsmTAid3C69FN54A3bdNTyIvvdedXJLCSUEEWk/a60Fp58eHjRP\nmgSffgoHHhjKSbfeCkuXJh1hSVNCEJH2t+aaMGYM/OtfYaTVsjI48sjw7OHqq8OUn9LulBBEJDll\nZXDYYfDSS/C3v8H3vgcnnxyarF58MXz+edIRlhQlBBFJnhnsvTc8+SQ8/jhsuSWce27o5HbuufDx\nx0lHWBKUEEQkPcxg2DB46CGYMQN23z3M5lZeDqecEh5KS2yUEEQknbbaCu64I7RIOvzwMHnPRhuF\nZw1vvJF0dEVJCUFE0m2TTeDGG0Mnt5NOgjvvhM03D62Tnn8+6eiKihKCiBSG9deHK64IZaPzzoPH\nHgv9GHbdNfyuTm5tpoQgIoVl3XXhwgtDYvjf/4VXXoFddoFttw3zQKuTW6spIYhIYVp7bTjzTHj3\nXbj2WvjoozBWUlUV/PGPsGxZ0hEWHCUEESlsnTvD8cfD7NkhEUAYXXXjjUOiWLIk2fgKiBKCiBSH\nsjIYPRpefjmUjtZbL8zHUFERSktffJF0hKmnhCAixaVDB9h3X3jmGXj00VBCOvvs0Jfh/PNh/vyk\nI0wtJQQRKU5msPPOMHVqaJ66yy4wceLKOaDffz/pCFNHCUFEil9NDdx9d5i1bdQouOYa2GADOPro\nMMCeAEoIIlJKNtsMbr4Z5syBE06A224L2w4+GF54IenoEqeEICKlp7wcrrwy9GU45xx4+GEYPDiM\nnfT44yXbyU0JQURK13rrwUUXhcRw8cUwcyYMHw7bbw9//3vJJQYlBBGRHj1g3LjQye2qq+CDD2Cf\nfWDQoFBWKpFObkoIIiIZXbqEAfRmz4ZbbgmJ4PDDwwB7kyYVfSc3JQQRkaY6dYIjjoBXX4V774Xe\nvUNv6A02gEsvhS+/TDrCWCghiIisSocOYXyk556DadNCi6QzzwwPpe+6K+no8k4JQUSkJWYwYgQ8\n8khIDjvtFCbrKTJlSQcgIlJQhg4NZaQipDsEEREBlBBERKReIgnBzC40s5fNbKaZTTWz7yYRh4iI\nrJTUHcIl7l7l7tXA34ELEopDRETqJZIQ3L3hTBXdgNLqHy4ikkKJtTIys4uAI4DPgZ2TikNERILY\n7hDMbJqZvZpl2Q/A3c9z9/WBycDJqznOGDOrNbPa+ZrpSEQkNuYJj+ZnZuXA/e5e2dK+NTU1Xltb\n2w5RiYgUDzOb4e41Le2XSMnIzAa4++z61X2BN6O8bsaMGZ+Y2dxWnrYP8EkrX1usdE2a0zXJTtel\nuUK6JuVRdkrkDsHM7gY2AeqAucDx7v7vmM9ZGyVDlhJdk+Z0TbLTdWmuGK9JIncI7n5QEucVEZFV\nU09lEREBSishXJ90ACmka9Kcrkl2ui7NFd01SbyVkYiIpEMp3SGIiMhqlFRCMLNLzOzN+oH17jWz\ndZKOKWlmdrCZvWZmdWZWVC0mcmVme5jZv8xsjpmNSzqepJnZ/5nZx2b2atKxpIWZrW9mj5nZG/X/\n34xNOqZ8KqmEADwMVLp7FTALOCfheNLgVeBA4ImkA0mSmXUErgZGAgOBw8xsYLJRJe5mYI+kg0iZ\nZcDp7r4ZsA1wUjH9d1JSCcHdp7r7svrVZ4G+ScaTBu7+hrv/K+k4UmAoMMfd33b3b4E/A/slHFOi\n3P0J4NOk40gTd/+Pu79Q//uXwBvA95KNKn9KKiE0cTTwYNJBSGp8D3i/wfo8iuh/dMk/M6sAtgSe\nSzaS/Cm6OZXNbBrwnSz/dJ67/7V+n/MIt36T2zO2pES5JoJl2aYmeJKVmXUH7gZOazKcf0EruoTg\n7t9f3b+b2ZHA3sAIL5E2ty1dEwHCHcH6Ddb7Ah8kFIukmJl1IiSDye5+T9Lx5FNJlYzMbA/gbGBf\nd1+cdDySKs8DA8ysv5mtARwK3JdwTJIyZmbAjcAb7n5Z0vHkW0klBOAqYC3g4fr5nK9LOqCkmdkB\nZjYP2Ba438ymJB1TEuobG5wMTCE8KLzD3V9LNqpkmdltwDPAJmY2z8yOSTqmFNge+BGwS/1nyEwz\n2zPpoPJFPZVFRAQovTsEERFZBSUEEREBlBBERKSeEoKIiABKCCIiUk8JQUqGmS1v0FRwZv3QA7ke\nYx0zOzH/0a04/qZm9oyZfWNmZ8R1HpFs1OxUSoaZfeXu3dt4jArg7+5emePrOrr78gj7rQeUA/sD\nn7n7pa2JU6Q1dIcgJc3MOtbPk/F8/TwZx9Vv725mj5jZC2b2ipllRj79FbBh/R3GJWY23Mz+3uB4\nV5nZUfW/v2tmF5jZk8DBZrahmT1kZjPMbLqZbdo0Hnf/2N2fB5bG/uZFmii6sYxEVqOLmc2s//0d\ndz8AOAb43N2HmNmawFNmNpUw8ukB7v6FmfUBnjWz+4BxhDk1qgHMbHgL51zi7jvU7/sIcLy7zzaz\nrYFrgF3y/SZFWksJQUrJ15kP8gZ2A6rM7Af16z2AAYTB7iaa2TCgjjAU9n+14py3w4rRMbcD7gzD\n4QCwZiuOJxIbJQQpdQac4u6NxnCqL/usCwx296Vm9i7QOcvrl9G49Np0n0X1PzsAC7MkJJHU0DME\nKXVTgBPqhzTGzDY2s26EO4WP65PBzoQHvQBfEgZIzJgLDDSzNc2sBzAi20nqx8x/x8wOrj+Pmdmg\neN6SSOvoDkFK3Q1ABfBC/dDG8wktfCYDfzOzWmAm8CaAuy8ws6fqJ55/0N3PNLM7gJeB2cCLqznX\naOBaMzsf6ESYpvOlhjuY2XeAWmBtoM7MTgMGFtMkLJJeanYqIiKASkYiIlJPCUFERAAlBBERqaeE\nICIigBKCiIjUU0IQERFACUFEROopIYiICAD/Dw2m+rbnIDJTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb5e4ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可见，一般随机生成的分类线，错误率很高。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 计算scores，更新权重"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，计算scores，得分函数与阈值0做比较，大于零则$\\hat y=1$，小于零则$\\hat y=-1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)    # 预测输出初始化\n",
    "loc_n = np.where(s < 0)[0]    # 大于零索引下标\n",
    "y_pred[loc_n] = -1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接着，从分类错误的样本中选择一个，使用PLA更新权重系数$w$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第一个分类错误的点\n",
    "t = np.where(y != y_pred)[0][0]\n",
    "# 更新权重w\n",
    "w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代更新训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更新权重$w$是个迭代过程，只要存在分类错误的样本，就不断进行更新，直至所有的样本都分类正确。（注意，前提是正负样本完全可分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第 0次更新，分类错误的点个数：22\n",
      "第 1次更新，分类错误的点个数： 7\n",
      "第 2次更新，分类错误的点个数：13\n",
      "第 3次更新，分类错误的点个数： 0\n"
     ]
    }
   ],
   "source": [
    "for i in range(100):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))\n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        t = np.where(y != y_pred)[0][0]\n",
    "        w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8FPX9+PHXOwEMyCEILZck2KIV\nEZDLCwWrImCr4gmieBbFImjRqtWKP+vVrz68akVBwYu2arWIFQTRWlFBOUSLeHCJRrACKljuJO/f\nH59dskk2yQzZ2ZndfT8fj30kMxlm3jvRfWc+788hqooxxhjjRV7YARhjjMkcljSMMcZ4ZknDGGOM\nZ5Y0jDHGeGZJwxhjjGeWNIwxxnhmScMYY4xnljSMMcZ4ZknDGGOMZ/XCDiDVWrZsqUVFRWGHYYwx\nGWXRokUbVLVVbcdlXdIoKipi4cKFYYdhjDEZRUTWeDnOmqeMMcZ4ZknDGGOMZ5Y0jDHGeJZ1NY1k\ndu3aRXFxMdu3bw87lIxUUFBA+/btqV+/ftihGGNClhNJo7i4mCZNmlBUVISIhB1ORlFVNm7cSHFx\nMR07dgw7HGNMyHKieWr79u3su+++ljD2gIiw77772lOaMQbIkaQBWMKoA7t3uWXqVCgqgrw893Xq\n1LAjMlGSE81Txhhvpk6FkSNh61a3vWaN2wYYPjy8uEx05MyTRtjy8/Pp3r07Xbp04cwzz2Rr/P9K\nHy655BKWLVsGwO23317hZ0ceeWRK4jS57YYbyhNG3Natbr8xEHLSEJHJIvKNiCyt5uf9RWSTiCyJ\nvW5Kd4yp0rBhQ5YsWcLSpUtp0KABDz/8sO9zPProo3Tu3BmomjTeeeedlMRpctsXX/jbb3JP2E8a\njwMDazlmrqp2j71uSUNMgbfpHn300axYsQKAe+65hy5dutClSxfuu+8+ALZs2cJJJ51Et27d6NKl\nC8888wwA/fv3Z+HChVx33XVs27aN7t27MzzWZtC4cWMAzj77bGbMmLH7WhdccAHPP/88paWlXHPN\nNfTu3ZuuXbvyyCOPpPZNmazQoYO//Sb3hJo0VPVN4NswY6gs3qa7Zg2olrfppipxlJSUMHPmTA45\n5BAWLVrElClTePfdd5k/fz6TJk3i/fff55VXXqFt27Z88MEHLF26lIEDK+bVO++8c/eTy9RKgQ0d\nOnR3ktm5cyevvfYagwcP5rHHHqNZs2YsWLCABQsWMGnSJFavXp2aN2Wyxm23QaNGFfc1auT2m+Ry\nreNA2E8aXhwhIh+IyEwROTjoiwXVpht/MujVqxcdOnTg4osv5q233mLIkCHsvffeNG7cmNNOO425\nc+dyyCGHMGfOHK699lrmzp1Ls2bNPF9n0KBBvP766+zYsYOZM2dyzDHH0LBhQ2bPns2TTz5J9+7d\nOeyww9i4cSPLly+v25syWWf4cJg4EQoLQcR9nTjRiuDVCfqPzCiKeu+pxUChqv5PRAYD04BOlQ8S\nkZHASIAOdXyODqpNN/5kkEhVkx57wAEHsGjRImbMmMH111/PgAEDuOkmb+WcgoIC+vfvz6xZs3jm\nmWcYNmzY7mv96U9/4sQTT6zbGzFZb/hwSxJe1fRHZrbew0g/aajqZlX9X+z7GUB9EWmZ5LiJqtpL\nVXu1alXrdPA1Smeb7jHHHMO0adPYunUrW7Zs4R//+AdHH300a9eupVGjRpx77rlcffXVLF68uMq/\nrV+/Prt27Up63qFDhzJlyhTmzp27O0mceOKJTJgwYfe/+eyzz9iyZUvq35QxOSQXOw5E+klDRFoD\n/1VVFZE+uCS3Mchr3nZbxX7qEFybbo8ePbjgggvo06cP4LrUHnroocyaNYtrrrmGvLw86tevz4QJ\nE6r825EjR9K1a1d69OhRpa4xYMAARowYwcknn0yDBg12n/vzzz+nR48eqCqtWrVi2rRpqX9TxuSQ\nDh1ck1Sy/VlLVUN7AX8F1gG7gGLgYuAy4LLYz0cDHwEfAPOBI2s7Z8+ePbWyZcuWVdlXk6efVi0s\nVBVxX59+2tc/z0p+76ExdZEp/w8+/bRqo0aqrqLhXo0apSbedN8DYKF6+dz2clAmvVKRNExVdg9N\nugT5QRyEID7cw7gHXpOGaDXF2EzVq1cvrbzc68cff8xBBx0UUkTZwe6hSZeiouRNPoWF8Pnn6Y4m\nHGHcAxFZpKq9ajsu0oVwY0zuycXicmVRvgeWNIzJAZk0AC2bR6V7/T1E+R5Y0jAmy2XaALRsHZXu\n5/cQ5XtgScOYLJdpM9dm66h0P7+HKN8DSxppIiKMGzdu9/bdd9/NzTffnPLr2JTpprIot49XZ/hw\nV/AtK3Nfo/BhWVd+fw9RvQeWNJKp3KMsBT3M9tprL1544QU2bNhQ53PVxKZMN5VFuX08l2TL78GS\nRmU33wxXXVWeKFTddh2fCurVq8fIkSO59957q/xs/fr1nH766fTu3ZvevXvz9ttv795/wgkn0KNH\nDy699FIKCwt3J51TTz2Vnj17cvDBBzNx4kQAmzLdJBVk+/jll0O9eq4JpV49t50KmVS49xprUL+H\ntN8rL4M5MulVp8F9ZWWqY8e6kTRjxybf3kN77723btq0SQsLC/X777/Xu+66S8ePH6+qqsOGDdO5\nc+eqquqaNWv0Zz/7maqq/vrXv9bbb79dVVVnzpypgK5fv15VVTdu3Kiqqlu3btWDDz5YN2zYsPs6\nla+rqvrCCy/oiBEjVFV1x44d2r59e926das+8sgj+oc//EFVVbdv3649e/bUVatWVYnfBvdltiAG\noI0aVXHwWfw1alTdY82UwX1+Y0317yGV9wobEV7O1wdeYqKIv+qYMFTLP7x///vf6y233FIhabRq\n1Uq7deu2+9W2bVvdvHmzduvWrcIHePPmzXcnjfHjx2vXrl21a9eu2rRpU503b16F61S+7rZt27R9\n+/a6fft2nTZtmp5zzjmqqnr66adrp06ddl+7qKhIZ82aVSV+Sxqmsvz85EkjP79u5y0sTH7ewsJU\nRJ1aYceayut7TRqRnrAwFCJw771w//3l++691+1PgSuvvJIePXpw4YUX7t5XVlbGvHnzaNiwYYVj\n3e+xqjfeeIM5c+Ywb948GjVqRP/+/dm+fXuN17Up002qlZb62+9VJhXuw441jOtbTaMyjdUwEiXW\nOOqoRYsWnHXWWTz22GO79w0YMIAHH3xw93Z83Y2+ffvy7LPPAjB79my+++47ADZt2kTz5s1p1KgR\nn3zyCfPnz9/9b23KdJMu+fn+9nuVSQXjsGMN4/qWNBLFE8b998PYsa6v29ixbjuFiWPcuHEVelE9\n8MADLFy4kK5du9K5c2cefvhhAMaPH8/s2bPp0aMHM2fOpE2bNjRp0oSBAwdSUlJC165d+f3vf8/h\nhx+++1zxKdOHJ+mfN2DAAN58802OP/74ClOmd+7cmR49etClSxcuvfRSSkpKUvI+TXYbOdLffq+i\nUDAOu7jtVSjX99KGlUmvOtc0xo+vWMOI1zhi9Yd02r59u+7atUtVVd955x3t1q1b2mOIs5qGqezp\np1Xr1avYll6vXjSnBfdTMA67uO1Xqq6PzXJbzvcMraoVaxiVt9Nk+fLlnHXWWZSVldGgQQMeeugh\nevfunfY4wGa5NVVl0my0fmLNpPeVSl5nubVCeDKVE0QICQOgU6dOvP/++6Fc25jahF0E9sNPrJn0\nvsKQMzWNbHuiSie7dyaZsIvAfviJNZPeVxhyImkUFBSwceNG+/DbA6rKxo0bKSgoCDuUtAhqdG0Q\nRdigeB3l7bcIG8T7CqJgHXZxO+pyonmqffv2FBcXs379+rBDyUgFBQW0b98+7DACF5+6Oj4TaXzq\naqjbZHF+zhtUDF5dfjlMmFC+XVpavv3QQxWPjcdzww2u6aZDB/fBmizOIN6Xn3P6idXPsbkoJwrh\nxngRVAE0k4qw9eolH5yXnw916YkdxPsK+15lG6+FcEsaxsTk5SUfiiPihuyk47xBxeBVTX0+6vJR\nEcT7CvteZRtbI9wYn4IqgGZSETaTRnmHfa9ylSUNY2KCKuxmUhE2KqO8vdzbIO9VJnVcSDsvIwAz\n6ZVsRLgxXnkdXRvkqOGwRxiPGlU+g21+ft2nOo8L4t4Gca+CHD0eZdiIcGOCY0XY4IR9bzOp40Iq\nZURNQ0Qmi8g3IrK0mp+LiDwgIitE5EMR6ZHuGI1JxkYNByfse2ujx2sWdk3jcWBgDT8fBHSKvUYC\nE2o41pi0sSKsf17b/v3c2yDqCZnUcSEMoSYNVX0T+LaGQ04Bnow1uc0H9hGRNumJzpjqhV2wzjTx\ngXhr1riW//hAvLoUuP2c049M6rgQCi+FjyBfQBGwtJqf/RPom7D9GtCrpvNZIdykS9gF60zid1lS\nL/c2yKVWM6njQqqQKYVwESkC/qmqXZL87GXgDlV9K7b9GvBbVV1U6biRuOYrOnTo0HNNssqUMSY0\nNrgv+jKiEO5BMbBfwnZ7YG3lg1R1oqr2UtVerVq1SltwxhhvbHBf9oh60pgOjIj1ojoc2KSq68IO\nyhi//BRsvc4yG9T1gxBE239O1hOiwEsbVlAv4K/AOmAX7qniYuAy4LLYzwX4M7AS+A+11DPUahom\ngvwMABs1Knk7fV0G2EVlAFpQA/GyoZ4QBWRKTSPVbHCfiRo/A8CCmGU2mwagmeBkS03DmIznZwBY\nsoRR0/5UX9+Y2ljSMCZgfgq2QcwyawVjk0qWNIwJ2G23QYMGFfc1aJC8YOt3ltmwZ4M1uceShjFp\nULl0WF0p8aijXF0jUb16bn9lXkdEDx8OEye6GoaI+zpxoi1favaMFcKNCVhQs6ZagdukkhXCjYmI\noGZNtQK3CYMlDWMCFtSsqVbgNmGwpGHMHvI6cjuoWVNvuw3q16+4r379uhe4balTUyMvIwAz6WUj\nwk06+B25HcSsqU8/rdqgQcXrN2hQt1HRubrUqbER4WGHYbJcECO3/QqiEG6F+NxlhXBjAhTEyG2/\ngiiEWyHe1MaShslYYban+x25HfaypEGc0wrxucmShslIQS316ZWfkdtBxTp4sL/9XthSp6ZWXgof\nmfSyQnhuCHKpT69GjVLNz3fXzc+vvggeVKxBnTcXlzo1VggPOwwTsExa6jOoWDPpHpjos0K4yWqZ\n1J4eVKyZdA9M9rCkYTJSFNrTvRa3g4o1CvfA5CAvbViZ9LKaRu4Isz3d78C2oGK1moJJFaymYUxw\nbGCbyTZW0/BLFe66Cz7+OOxITAawgW0mV1nSiFu1Cn73O+jcGY48EiZNgs2bw47KRJQVoU2usqQR\n95OfQHEx3H03bNrkRl+1bg3nnw9vvGF9GE0FQc0wa0zUWdJI9OMfw7hxsHQpvPsujBgB06bBscdC\np05w663w5ZdhR2kiQqTmbWOykRXCa7N1K/zjHzB5Mrz+uvtkOOEEuOgiOOUUKChI3bVMxrBCuMk2\nVghPlUaNYPhweO01V/e46Sb45BMYOhTatoXRo2Hx4uRDc03WskK4yVWWNPzo2BFuvhlWr4ZXX4VB\ng+DRR6FnT+jeHe6/HzZsCDtKkwZWCDe5KtSkISIDReRTEVkhItcl+fkFIrJeRJbEXpeEEWcVeXlw\n/PFuCPC6dfDQQ7DXXnDlle7p44wzYMaM9K3GY9LO72hsWxbVZA0vIwCDeAH5wEpgf6AB8AHQudIx\nFwAP+jlvqCPCP/xQ9aqrVFu1ckOE27ZVve461U8/DS8mExg/y7Lasqgm6oj6iHAROQK4WVVPjG1f\nD6CqdyQccwHQS1VHez1vJEaE79wJL7/siuczZ7rl3Pr2hQsvhDPPhCZNwo3PpJUVzU0myIRCeDsg\nsf9qcWxfZaeLyIci8ncR2S/ZiURkpIgsFJGF69evDyJWfxo0gCFD4KWXXBfdP/4R1q+Hiy+GNm1c\nz6u5c614niOsaG6ySZhJI1mv9sqfoi8BRaraFZgDPJHsRKo6UVV7qWqvVq1apTjMOmrTBn77Wzc9\nyTvvwLBh8NxzcMwxcOCBcMcd8NVXYUcZqFxvz/dbNM/1+2WiLcykUQwkPjm0B9YmHqCqG1V1R2xz\nEtAzTbGlnggccYSbnuTrr+GJJ1zR/He/c58egwfD3/8OO3bUfq4MEvayrFHgp2hu98tEnpfCRxAv\noB6wCuhIeSH84ErHtEn4fggwv7bzZtzU6CtWqN54o2r79q5C2qKF6pgxqkuWhB1ZSkRhWdYo8Fo0\nt/tlwkJdC+Eicgjur/t2wEzgWlX9Lvaz91S1T10TlogMBu7D9aSarKq3icgtseCni8gdwMlACfAt\nMEpVP6npnJEohO+J0lKYMwemTHEj0HfuhEMPdfWPc86BFi3CjnCP2JKk/tj9MmHxWgivKWm8BdwK\nzAcuAS4ETlbVlSLyvqoemsqAUyVjk0aib7+Fv/zFJZDFi11h/dRTXQI5/njIzw87Qs+s55A/dr9M\nWFLRe6qxqr6iqt+r6t3AaOAVETmcqgVrk0otWrjpSRYtgvffh8suc9OYDBzoPlVuvBFWrgw7Sk9s\nSVJ/7H6ZqKspaYiINItvqOq/gNOBp4DCoAMzMfHpSb76yvW6OuQQ1+Pqpz+F/v1dQX3LlrCjrNbw\n4TBxovtLWcR9nTjR7TdV2f0yUVdT89Q5wCpVnV9pfwfg96r6qzTE51tWNE/V5quv4Mkn3eDBFSvc\nYMGzz3aDB484wuboNsb4VueaRqbKiaQRpwpvv+2Sx7PPuieOAw90tY/zznNjRIwxxoNMGBFu6krE\nTU8yebIb+zF5MrRqBddeC/vtB7/8ZXlPLGOMSQFLGtmicWPXPDV3Lnz6qRuFvngxnHYatG8Pv/mN\nW5EwRfyMWrYRzsZkD2ueymYlJTB7tuu6++KLsGsX9O7tksuwYbDPPnt02vio5a1by/c1apS8YOvn\nWGNMeFJW0xCRA4AJwI9VtYuIdMWN17g1NaGmliWNamzY4D7BJ0+GDz90y9SedppLID//uXsM8MjP\nWAIbd2BMZkhl0vg3cA3wSHxAn4gsVdUuKYk0xSxp1ELVjf2YPNkNIPzuO/cJfsEF7lVUVOsp/Ixa\nthHOxmSGVBbCG6nqe5X22ZJ0mUoEevSABx+EtWvhb39zPa5uucUtZ3vcce6JJLE9qRI/s7basqjG\nZBcvSWODiPyE2ChwETkDWBdoVCY9Cgrc+I5Zs1xb0R/+4L6ee67rrnvZZfDee1UeFfyMWrYRzsZk\nmdpmNMQtxzoH2Ap8BbwFFHqZDTGMV8bNchs1paWqb7yhOmKEasOGborVzp1V775b9euvdx/mddZW\nv8caY8JBKpZ7FZE84AxVfVZE9gbyVPWHwDNZHVhNI4U2b3aDBidPhnnzoF49OOkkN3hw0CCoXz/s\nCI0xKZKSmoaqluEmKkRVt0Q9YZgUa9oULrnErTi4bBlcdRXMnw+nnOIGD15zjVuR0BiTM7zUNF4V\nkatFZD8RaRF/BR6ZiZaDDoL/+z+35vn06W6Oq/vug86dy1ck3Lw57CiNMQHz0uV2dZLdqqr7BxNS\n3VjzVBp98w08/bRrvvroI2jYEM44wzVfHXOMr7Efxphw2YSFJn1UYeHC8rEfmze77rsXXgjnn2/9\na43JAKkc3Dci2X5VfXIPYwuUJY2QbdvmJkmcPNktHCXiVhu86CK3+mBBQdgRGmOSSOXgvt4Jr6OB\nm3HrdhtTVcOGbk3zOXNg9WoYPx4++8zNddWmDfz6125Fwix7wjUmV9SaNFT1ioTXr4BDgQbBh2Yy\nXlGRSxqrVrkkctJJ7gmkVy+3IuF998H69RX+ic2Ia0y07UmlcivQKdWBmCyWl+emJ3n6aVi3DiZM\ncM1UV10F7dq54vnLL/OXJ0sYOdJNcKjqvo4caYnDmCjxUtN4idgUIrgk0xl4TlWvDTi2PWI1jQyy\ndKmbtv2pp2D9ev6b34bJpeczhQtZzgG7D7MZcY0JXioL4f0SNkuANapaXMf4AmNJIwPt3AkzZjB9\nyGQGM4N6lPIWRzGZi3iOM9kiTWxGXGMClspC+GBV/Xfs9baqFovIH1MQozFOgwZw6qmMKZxOe4q5\nhv9jXzYymYtZRxueaRRbkdCK58aEzkvSOCHJvkGpDsSY226DHxq15m6uoTPLOIJ3eDb/HE4tfd4N\nFjzgALj9diiO7IOuMVmv2qQhIqNE5D/AgSLyYcJrNfBhKi4uIgNF5FMRWSEi1yX5+V4i8kzs5++K\nSFEqrmuiafhwtwxsYSGICOsKj2CvJyZSf8M6eOIJt9b5DTe4AwYNgueegx07wg7bmJxSbU1DRJoB\nzYE7gMQP9B9U9ds6X1gkH/gM9yRTDCwAhqnqsoRjLge6quplIjIUGKKqZ9d0XqtpZLmVK+Hxx10S\n+fJLaNHCZZuLLnLdeI0xe6TONQ1V3aSqn6vqMFVdA2zD9aJqLCKpmBeiD7BCVVep6k7gb8AplY45\nBXgi9v3fgeNERFJwbZOpfvITt1jU6tVu8agTTnCPJ4ce6lYk/NOfYOPGsKM0JmvVWtMQkV+KyHJg\nNfBv4HNgZgqu3Q74MmG7OLYv6TGqWgJsAvZNwbVNpsvPhwED3HK1a9e65WtFYMwYaNvWrUj4yitQ\nWhp2pMZkFS+F8FuBw4HPVLUjcBzwdgquneyJoXJbmZdjEJGRIrJQRBaurzTC2OSAFi3KpydZsgRG\njXLzXg0a5IaV33gjrFgRdpTGZAUvSWOXqm4E8kQkT1X/BaSi8bgY2C9huz2wtrpjRKQe0AyoUk9R\n1Ymq2ktVe7Vq1SoFoZmM1a2bm57kq6/g73+Hrl3hjjugUyfo18/VQrZsCTtKYzKWl6TxvYg0BuYC\nU0Xkftwgv7paAHQSkY4i0gAYCkyvdMx04PzY92cAr2u2zeVugrHXXnD66fDyy/DFFy5xrFsHF1wA\nrVuXr0ho/zkZ44uXEeF744rgecBw3F/7U2NPH3W7uMhg4D4gH5isqreJyC24Bc6ni0gB8BRuksRv\ngaGquqqmc1rvKVMtVXj7bTd1yTPPuCeOAw90636MGOFm4TUmR6V0ESYRKQQ6qeocEWkE5Ed1vXBL\nGsaT//3PjfOYMsWNNs/Ph4EDXdfdX/zCjVI3JoekbBoREfkVrrvrI7Fd7YBpdQvPmJA1buyeMN58\nEz79FH77W3j/fdek1a6dm4H3P/8JO0pjIsdLTePXwFHAZgBVXQ78KMigjEmr+PQkX3wBM2ZA//7w\n5z+7Inrv3vDQQ/Ddd2FHaUwkeEkaO2KD74DdvZisemiyT35++fQka9fC/ffDrl2uO2+bNm5Fwldf\nxabcNbnMS9L4t4j8DmgoIicAzwEvBRuWMSFr2dINFFyyBBYvhl/9yg0WHDAAOnZ0KxKuXh12lMak\nnZekcR2wHvgPcCkwA7gxyKCMiZRDD3XTk6xd63pdHXSQm8pk//3h5z93KxJu3Rp2lMakRU0TFnZQ\n1S/SHE+dWe8pkxZffglPPunWPF+1Cpo2haFDXe+rPn3clCbGZJBU9J7a3UNKRJ5PSVTGZIv99nPT\ntC9fDm+8AUOGuCeOww+Hgw+Gu++G//437CiNSbmakkbin0r7Bx2IMRkpL89NT/L4427E+aRJ0Lw5\nXHON67p7yinw4ouuoG5MFqgpaWg13xtjkmna1E1P8vbb8PHHMG4cvPcenHqqW0Dq6qth2bLaz2NM\nhNWUNLqJyGYR+QHoGvt+s4j8ICKb0xWgMRnpZz+DP/7R1T5eegmOOsp14T34YNeE9cgjsGlT2FEa\n41tNizDlq2pTVW2iqvVi38e3m6YzSGMyVr16blqSF15wM+/ec4+bwuSyy9zYj/POg9dft7EfJmN4\n6XJrjEmFH/2ofHqS995zM+6+9BIcd5xbkfCWW2DNmrCjNKZGljSMSTeR8ulJ1q2DqVPhpz91AwY7\ndnRL2P71r7BtW9iRGlOFJQ1jwtSwYfn0JKtXu8SxYoXb16YNXH45LFxo636YyLCkYUxUFBW5pLFy\npVuu9he/cFO39+7tJk+8916w5YxNyCxpGBM1eXnl05OsWwcPPwyNGsFvfgNt27rp2//5TyhJxQKa\nxvhjScOYKNtnH7j0Unj3XVi6FMaOdYtG/fKXblT6dde59UCMSRNLGsZkivj0JF99BdOmuTmu7r7b\njQk56ih47DH4IZILaposYknDmExTv3759CTFxXDXXW6RqEsugdatXVfeN9+04rkJhCUNYzJZ69Zu\nepKPPoJ58+Dcc91Awn79oFMnuO02l1iMSRFLGsZkA5Hy6Um+/tpN296hA9x4o/s6cCA8+yzs2BF2\npCbDWdIwJts0alQ+PcnKlS5xLFsGZ5/tel9dcQW8/37YUZoMZUnDmGy2//5uepLVq2H2bLdc7aRJ\n0KOHW5HwgQdg48awozQZxJKGMbkgP798epJ16+DPf3b7xo51Tx9nnQUzZ0JpadiRmoizpGFMrmne\nvHx6kg8+cN+//joMHgyFheUrEhqTRChJQ0RaiMirIrI89rV5NceVisiS2Gt6uuM0JuvFpydZuxae\nfx66d4c774QDDoBjjnErEv7vf2FHaSIkrCeN64DXVLUT8FpsO5ltqto99jo5feEZk2MaNIDTTnPT\nk3z5pUsc//0vXHih69Z78cVuRUIb+5HzwkoapwBPxL5/Ajg1pDiMMZW1bQvXXguffAJvvQVDh7ru\nun37utHnd97pnkxMTgorafxYVdcBxL7+qJrjCkRkoYjMFxFLLMakk4ibnuTRR13xfMoU99Rx/fVu\n3quTTnJNWjt3hh2pSSPRgB43RWQO0DrJj24AnlDVfRKO/U5Vq9Q1RKStqq4Vkf2B14HjVHVlkuNG\nAiMBOnTo0HONrX5mTHCWL3e1jieecPNgtWwJw4fDRRe5GonJSCKySFV71XpcUEmjxouKfAr0V9V1\nItIGeENVD6zl3zwO/FNV/17Tcb169dKFCxemLlhjTHKlpW7xqMmT3TxYO3dCz56uDnLOOa6XlskY\nXpNGWM1T04HzY9+fD7xY+QARaS4ie8W+bwkcBSxLW4TGmJrl55dPT7J2rRsoWFoKo0e7VQeHDXMD\nCm3sR1YJK2ncCZwgIsuBE2LbiEgvEXk0dsxBwEIR+QD4F3CnqlrSMCaK9t23fHqSxYth5EiYNQtO\nPNGte37TTbBqVdhRmhQIpXlCmpjhAAAPvElEQVQqSNY8ZUxEbN8O06e7AvqsWa67bv/+rvZx+ulu\njiwTGVFvnjLGZLuCgvLpSdasgVtvdWNARoxwvbBGjoT5823sR4axpGGMCd5++5VPT/Lvf7uBhFOn\nwhFHQOfObiGpr78OO0rjgSUNY0z6iJRPT/L1124MSIsW8NvfQvv2cPLJbinbXbvCjtRUw5KGKVe5\nmSCMZoMoxGDSo0mT8ulJPvnErUC4YAEMGeISyLhxbkVCEymWNIxz881w1VXlH9Kqbvvmm3MrBhOO\nAw9005N8+aWb/6pvX9eFt0sXOOwwePhh+P77sKM0WNIw4D6cv/8e7r+//EP7qqvc9vffp+ev/SjE\nYMJXr1759CRr17oZeLduhVGj3NiPc8+F116DsrKwI81Z1uXWOIkf0nFjx7r/aUVyJwYTPaqwaJEb\nef6Xv8CmTW7djwsvhPPPh6KisCPMCpGeRiRIljTqQBXyEh4+y8rS/2EdhRhMdG3b5grlkye7Jw5V\nOO44N/ZjyBBo2DDsCDOWjdMw/sT/yk+UWF/IlRhMtDVs6KYnefVVt+75//t/sHKlmzCxTRvXjLVg\ngf03EyBLGqZis9DYse6v+7FjK9YXciEGk1kKC930JCtXuuVqf/lL15W3Tx845BC45x745puwo8w6\nljSMa/7ZZ5+K9YN773Xb++yTnuahKMRgMlNeHhx7LDz1lBv78cgj0Lix67Lbrp1rtnrpJSgpCTvS\nrGA1DVNOteKHc+XtXInBZIdly9y8V08+6Z44WreG885zBfSDDgo7usixmobxr/KHcy5+WFfuyhlG\n104b4Jga8elJiotd8fyww1yTVefOcOSRMGkSbN4cdpQZx5KGiY6wB/f17+8WEYonirIyt92/f3qu\nD+Hfg2xUvz6ccopLHMXFLpF8/72bMLF1a9dt9403bOyHR5Y0TDSEPbivrMz1/1+ypDxx9Ozptjdt\nSs8HStj3IBe0bu2mK/noIzfD7nnnuWRy7LHQqVP5TLymeqqaVa+ePXuqyVBlZapjx6q6j0f3GjvW\n7U+H0lLV7t0rXr97d7c/XcK+B7loyxbVp55SPfZYd79FVAcMUP3b31S3bQs7urQBFqqHz1grhJto\nCXtwX1mZW8Y0rrS0YjzpEPY9yGWrV7tuu48/Dl984dY5P+ccN3jw0EOz+vdghXDj+CmqBlUErrxG\ndHVrRqvClVdW3Hfllelrlok3SSVKrHGkgw1wDFfHjm7A4OrVbgDhwIFu+vaePaF7d9dUuGFD2FGG\ny8vjSCa9rHkqwfjxFZs24k0f48dXPbZfv4pNMfGmmn796hZDYaFqy5aqJSVuu6TEbRcWVjyurEz1\nsMNc88CYMW57zBi3fdhhwTfPJDZNxe9D5e2gJTZNxX9vlbdN+n37repDD6n26uV+F/Xrq55+uurL\nL6vu2hV2dCmDx+Ype9LIVuqjqBpUEbi0FLZscX+ZtW7ttlu3dttbtlT/xBGGvDxo1sz9Nblokdte\ntMhtN2uWniYqG+AYTc2bl09P8uGHMHq0W33wpJPcqPTrr4fPPgs7yvTxklky6WVPGgn8FFWDKgLH\nnywSz5v45FE53vjTRfwVf+pIl8rvN51F8LjK79eeMKJnxw7V559X/cUvVPPy3H+rffuqPvaY6ubN\nYUe3R7BCuAH8FVWDKgKXlrp1EuJKSipeZ0/jNSYK1q51U5hMmQKffgp77w1nneVGnvftmzH//Voh\nPNtVTvbJkn+8SSpRdUVVv0Vgr0XzeJNUonhTVbJ4/RTCvcbg5V75PWeQ/MRrwte2LVx7LXz8sVu6\nduhQeO45txZ6165ZN+eVJY1M5GXUcGINo7ZZYxNrGN27uw/07t0r1jgSeR05XVoKTZu6GkbLlu5/\nnpYt3XbTphUTh6qrHTzwAIwZ4845Zozbbtas6gen1xj8jLC2EeGmLkTc9CSPPuomTnz8cTjzzIpP\n2VnAkkam8Vrg9lNU9VME9lM0z8srb4YaOtRtDx3qtvPzq543/hfZ1Klue+pUt11SUvG8XmOIQmcA\nP/zEa6Jt773d9CQ33RR2JKnnpfCRSa+cKIT7KXD7Kap6LQL7KZqXlamOHl3x2NGjk8dRUqK6774V\nj9133+RFc68xRKEzgB82ItyEhCgXwkXkTOBm4CCgj6omrVyLyEDgfiAfeFRV76zt3JErhGtAU337\nKS57VVZW9a//6grhJSVuIri4Xbuqfwz3E+uuXdCgQfn2zp0Vr7Mn5w2qM0BQv1vrDGBCEPVC+FLg\nNODN6g4QkXzgz8AgoDMwTEQ6pye8FAmqfbpfv+TF5X799vycftrz+/WDJk0q7mvSJPn1/cR69NFV\n13hu2NDtr+ymm5Kft3JzQFCdAYL63fqJ15gweHkcCeoFvAH0quZnRwCzEravB66v7ZyRaZ4KanRv\n4riH+HiHytt++RkNvWuXakGB+1lBQfLtPYl1507V/Hz3s/z85Nt+z+vnd+DnHgT1u7UR4SZEeGye\ninJZvx2QOEdxMXBYsgNFZCQwEqBDhw7BR+ZFvPAMrpB5//3u+8TC9J7Iz3ejUydMcL2Q4s0zLVu6\n/XvSRBUvfMcLv/FzJBbG4+rVg9693ejY7dvLm44KCtz+xOYiP7HWr++W5vzqK9ckFG+iys93+xOb\nqPLz3UI6y5ZVPW/nzuXnra4zAHjvDNCzZ9XOAEH9bv3Ea0xIAqtpiMgcoHWSH92gqi/GjnkDuFqT\n1DRidY8TVfWS2PZ5uPrHFTVdN5I1jSDap4OqaXhtz8+0mobX2oOfuk5Qv9ugaiXG1CD0moaqHq+q\nXZK8XvR4imJgv4Tt9sDa1EcaIFX3V2OisWOrb5+uvL+648rKoFel322vXsnb3v2c02t7flmZe6pI\n1Lt39cd6jbWsDPr0qbivT5/q39e4cRX3jRuX/P35WcbW67HqcyCiH7bsrokyL21YQb2ouaZRD1gF\ndAQaAB8AB9d2zkjVNNq1c+3RV1zhtq+4wm23a1e1fdrrjLR+2t6DOKffY1u3Tn5s69Z7ft6g2v69\n3q+wZ+Q1JgBEeZZbERkiIsW4YvfLIjIrtr+tiMwAUNUSYDQwC/gYeFZVPwoj3j2iWj7iee5ctz13\nrtsuLa34F6n6GNTldSBeEOf0e6xIeXPR0Ue77XhPqPz8PR9gGMRssH7ulzG5zEtmyaRXZJ40VP0P\ngvMzqMvLQLwgzun32MQnrPgr/uRV1xhSPRus30GTYc/Ia0wKEeXBfUGKXCHc72CxVBdWgzhnJsbg\nlZ9YM+l9GVOL0AvhBn/F5XhzSKK6DuoK4px7EkOYS7j64ed+ReHexuOoaduYVPPyOJJJr8g0T4Vd\n2I3CQLFMKhj7uV9RuLeq/pbzNaYWZMHgvszmd7BYqgd12UAxf/zcryjcW00o3IO7fuJU+GpjO0ww\nrKYRNL+DxVI9qCuIc/q9/pVXunUx4saMgfvui+aHmp/7FYV7G08UcXUdlW5ylteahiUNEzwrGAfH\n7q1JESuEB8mKj95FpWCcjezemhBY0vDLluP0LrH5pLYlZ40/dm9NSKwQ7ocVH/2JQsE4W9m9NSGx\nmoZfVnz0L+yCcTaze2tSxArhQYpC8dE+LIwxKWSF8KBEofhodRVjTEgsafgRheJjYl3FZmM1xqSZ\nFcL9iELxMailRo0xxgOraeyJKNQTolBXMcZkDatpBCns5TijUFcxxuQkSxqZJgp1FWNMzrKaRqaJ\nQl3FGJOzrKaRqaJQVzHGZA2raWS7sOsqxpicZEnDGGOMZ5Y0jDHGeGZJwxhjjGeWNIwxxnhmScMY\nY4xnljSMMcZ4lnXjNERkPbCmDqdoCWxIUTjZwu5JVXZPqrJ7UlUm3ZNCVW1V20FZlzTqSkQWehng\nkkvsnlRl96QquydVZeM9seYpY4wxnlnSMMYY45kljaomhh1ABNk9qcruSVV2T6rKuntiNQ1jjDGe\n2ZOGMcYYzyxpVCIid4nIJyLyoYj8Q0T2CTumKBCRM0XkIxEpE5Gs6g3ih4gMFJFPRWSFiFwXdjxR\nICKTReQbEVkadixRISL7ici/ROTj2P83Y8OOKVUsaVT1KtBFVbsCnwHXhxxPVCwFTgPeDDuQsIhI\nPvBnYBDQGRgmIp3DjSoSHgcGhh1ExJQA41T1IOBw4NfZ8t+KJY1KVHW2qpbENucD7cOMJypU9WNV\n/TTsOELWB1ihqqtUdSfwN+CUkGMKnaq+CXwbdhxRoqrrVHVx7PsfgI+BduFGlRqWNGp2ETAz7CBM\nZLQDvkzYLiZLPghMcESkCDgUeDfcSFIjJ9cIF5E5QOskP7pBVV+MHXMD7hFzajpjC5OX+5Ljki2P\naN0PTbVEpDHwPHClqm4OO55UyMmkoarH1/RzETkf+AVwnOZQn+Ta7ouhGNgvYbs9sDakWEzEiUh9\nXMKYqqovhB1PqljzVCUiMhC4FjhZVbeGHY+JlAVAJxHpKCINgKHA9JBjMhEkIgI8BnysqveEHU8q\nWdKo6kGgCfCqiCwRkYfDDigKRGSIiBQDRwAvi8issGNKt1gHidHALFxh81lV/SjcqMInIn8F5gEH\nikixiFwcdkwRcBRwHvDz2OfIEhEZHHZQqWAjwo0xxnhmTxrGGGM8s6RhjDHGM0saxhhjPLOkYYwx\nxjNLGsYYYzyzpGFMJSJSmtBNcklsGgi/59hHRC5PfXS7z/8zEZknIjtE5OqgrmNMZdbl1phKROR/\nqtq4jucoAv6pql18/rt8VS31cNyPgELgVOA7Vb17T+I0xi970jDGAxHJj621siC21sqlsf2NReQ1\nEVksIv8Rkfist3cCP4k9qdwlIv1F5J8J53tQRC6Iff+5iNwkIm8BZ4rIT0TkFRFZJCJzReRnleNR\n1W9UdQGwK/A3b0yCnJx7yphaNBSRJbHvV6vqEOBiYJOq9haRvYC3RWQ2btbbIaq6WURaAvNFZDpw\nHW5dlu4AItK/lmtuV9W+sWNfAy5T1eUichjwEPDzVL9JY/aEJQ1jqtoW/7BPMADoKiJnxLabAZ1w\nkxjeLiLHAGW4qdJ/vAfXfAZ2z4p6JPCcm74IgL324HzGBMKShjHeCHCFqlaYcyvWxNQK6Kmqu0Tk\nc6Agyb8voWJzcOVjtsS+5gHfJ0laxkSC1TSM8WYWMCo23TUicoCI7I174vgmljCOxRWnAX7ATXwZ\ntwboLCJ7iUgz4LhkF4mtubBaRM6MXUdEpFswb8kY/+xJwxhvHgWKgMWxaa/X43ouTQVeEpGFwBLg\nEwBV3Sgib4vIUmCmql4jIs8CHwLLgfdruNZwYIKI3AjUxy0r+0HiASLSGlgINAXKRORKoHO2LPRj\nosu63BpjjPHMmqeMMcZ4ZknDGGOMZ5Y0jDHGeGZJwxhjjGeWNIwxxnhmScMYY4xnljSMMcZ4ZknD\nGGOMZ/8fqspA78ZTvcIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb534d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实，PLA算法的效率还算不错，只需要数次更新就能找到一条能将所有样本完全分类正确的分类线。所以得出结论，对于正负样本线性可分的情况，PLA能够在有限次迭代后得到正确的分类直线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. PCA优化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "刚刚我们的数据是线性可分的，可以使用PCA来得到分类直线。但是，如果数据不是线性可分，即找不到一条直线能够将所有的正负样本完全分类正确，这种情况下，似乎PCA会永远更新迭代下去，却找不到正确的分类线。所以，对于数据不可分的情况，应该要对PCA进行改进、优化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PCA优化的基本做法很简单，就是如果迭代更新后分类错误样本比前一次少，则更新权重系数$w$；没有减少则保持当前权重系数$w$不变。也就是说，可以把条件放松，即不苛求每个点都分类正确，而是容忍有错误点，取错误点的个数最少时的权重系数$w$。通常在有限的迭代次数里，都能保证得到最佳的分类线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集存放在'../data/'目录下，该数据集包含了100个样本，正负样本各50，特征维度为2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv('./data/data2.csv', header=None)\n",
    "# 样本输入，维度（100，2）\n",
    "X = data.iloc[:,:2].values\n",
    "# 样本输出，维度（100，）\n",
    "y = data.iloc[:,2].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据分类与可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们在二维平面上绘出正负样本的分布情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2UHHWd7/H3dx5CMiGEp7mKhExA\nUYSYDJPhUSFZgxBWBR8Q4caL4cgODyoBgaOsVxLdG12vXmMCSyDgenSTddEIXPQSYNFlDRDQCSSo\nBAUxCVlQkkgCJCQmme/9o6o7PT0909UzVVPV3Z/XOXWm62F+9avupL9T9a3ft8zdERERAWhIuwMi\nIpIdCgoiIpKnoCAiInkKCiIikqegICIieQoKIiKSp6AgdcPM/t7Mbo972whtuZm9LY62RJJmGqcg\n1cjMZgHXAG8FXgXuAq53961p9qsUM3PgaHd/rsS6h4CTgd2AA88CPwLmu/uuobYvUimdKUjVMbNr\ngK8D1wFjCb5U24B/N7MR/fxO0/D1sGKfcfcxwGEEge4C4F4zs3S7JfVIQUGqipkdAHwZ+Ky73+fu\nu919HXA+QWD4RLjdXDNbZmZLzOxVYFa4bElBWxeZ2Xoz22JmXzKzdWZ2RsHvLwlfTwgvAX3SzDaY\n2WYz+2JBOyea2Uoz22pmL5nZTf0Fp4G4+3Z3fwg4BzgFeH+59s3sF+GvrzGz183s42Z2kJn91Mw2\nmdkr4etxlfZH6pOCglSbU4GRwJ2FC939dWA58L6CxecCy4ADgaWF25vZscDNwEyCv9DHAoeX2fd7\ngHcA04EbzOyd4fK9wNXAoQRf5tOBKyo8rsJj2QB0A6eVa9/dTw+3mezu+7v7HQT/r79LECTHA28A\nNw22P1JfFBSk2hwKbHb3PSXWvRSuz1np7ne7e4+7v1G07XnAT9z9YXf/K3ADwTX9gXzZ3d9w9zXA\nGmAygLuvcvfH3H1PeNZyKzC18kPr5UXg4MG07+5b3P3H7r7D3V8D5sXQH6kTWb7OKlLKZuBQM2sq\nERgOC9fnvDBAO28pXO/uO8xsS5l9/6ng9Q5gfwAzezvwLaATaCH4f7WqTFvlHA48Opj2zawFmA/M\nAA4KF48xs0Z33zvEfkmN05mCVJuVwC7gI4ULzWw0cDbws4LFA/3l/xKQv85uZqOAQwbZp0XAMwR3\nAB0A/D0w6CSxmR0BTAFWDLL9awguc50Ubp+7xKTEtZSloCBVxd23ESSabzSzGWbWbGYTCG7j3Aj8\nS8SmlgEfNLNTw6Ttlxn8l+YYgttiXzezY4DLB9OImbWY2VTg/wK/BO6N2P6fgaOK+vMGsNXMDgbm\nDKY/Up8UFKTquPv/Jvhr+ZsEX5aPE1wKmh713n53/y3wWeDfCM4aXgNeJjgLqdS1wH8P27gNuKPC\n37/JzF4j+HL/NvBjYIa790Rsfy7wvfDupPPDNkYRXEp7DLiv0gOS+qXBayKAme0PbCW4RPPHtPsj\nkhadKUjdMrMPhpdsRhOcdfwaWJdur0TSpaAg9excgls/XwSOBi5wnTpLndPlIxERydOZgoiI5FXd\n4LVDDz3UJ0yYkHY3RESqyqpVqza7e2u57aouKEyYMIHu7u60uyEiUlXMbH2U7XT5SERE8hILCmb2\nDjNbXTC9amZXFW1jZrbQzJ4zs6fMrCOp/oiISHmJXT5y998B7QBm1gj8F8HTsQqdTXAr4NHASQQ1\nXk5Kqk8iIjKw4copTAf+4O7F17TOBb4f3hv+mJkdaGaHuftLlTS+e/duNm7cyM6dO+Pqb10ZOXIk\n48aNo7m5Oe2uiEjKhisoXAD8oMTyw+ld3nhjuKxXUDCzLqALYPz48X0a2bhxI2PGjGHChAnoCYaV\ncXe2bNnCxo0bOfLII9PujoikLPFEc1iB8hyCKpZ9VpdY1mc0nbsvdvdOd+9sbe17R9XOnTs55JBD\nFBAGwcw45JBDdJYlIsDw3H10NvCEu/+5xLqNwBEF8+MISg5UTAFh8PTeicRn6VKYMAEaGoKfS5eW\n+41sGY6gcCGlLx0B3ANcFN6FdDKwrdJ8gohIVixdCl1dsH49uAc/u7qqKzAkGhTCxwK+j4KHrJvZ\nZWZ2WTh7L/A88BxBnfhBP+w8bY2NjbS3tzNx4kQ+9rGPsWPHjorbuOSSS3j66acB+OpXv9pr3amn\nnhpLP0UkOV/8IhT/19+xI1heLaquIF5nZ6cXj2heu3Yt73znO1PqUWD//ffn9ddfB2DmzJlMmTKF\nz33uc7G0Nxyy8B6KVLuGhuAMoZgZ9PT0XT6czGyVu3eW264uRzQnfc3vtNNO47nnngPgW9/6FhMn\nTmTixIl8+9vfBmD79u28//3vZ/LkyUycOJE77ggepDVt2jS6u7v5whe+wBtvvEF7ezszZ84EgiAB\n8PGPf5x77703v69Zs2bx4x//mL1793LddddxwgknMGnSJG699dZ4D0pEyipxc+SAyzPJ3atqmjJl\nihd7+umn+yzrz5Il7i0t7kE8D6aWlmD5UIwePdrd3Xfv3u3nnHOO33zzzd7d3e0TJ070119/3V97\n7TU/9thj/YknnvBly5b5JZdckv/drVu3urv71KlT/Ve/+lWv9orbv/POO/2iiy5yd/ddu3b5uHHj\nfMeOHX7rrbf6P/zDP7i7+86dO33KlCn+/PPPR+5/Je+hiJSW1PdLHIBuj/AdW3dnCkld88v9Zd/Z\n2cn48eP51Kc+xcMPP8yHP/xhRo8ezf77789HPvIRVqxYwbve9S4efPBBPv/5z7NixQrGjh0beT9n\nn302P//5z9m1axfLly/n9NNPZ9SoUTzwwAN8//vfp729nZNOOoktW7bw7LPPDu2gRKQiM2fC4sXQ\n1hZcMmprC+bDE/6qUHVVUodqw4bKlkc1atQoVq9e3WuZ95Ovefvb386qVau49957uf766znzzDO5\n4YYbIu1n5MiRTJs2jfvvv5877riDCy+8ML+vG2+8kbPOOmtoByIiQzJzZnUFgWJ1d6YwnNf8Tj/9\ndO6++2527NjB9u3bueuuuzjttNN48cUXaWlp4ROf+ATXXnstTzzxRJ/fbW5uZvfu3SXbveCCC/ju\nd7/LihUr8kHgrLPOYtGiRfnf+f3vf8/27dvjPygRqWl1d6Ywb15w33DhJaSWlmB53Do6Opg1axYn\nnngiENxyevzxx3P//fdz3XXX0dDQQHNzM4sWLerzu11dXUyaNImOjg6WFmXCzzzzTC666CLOOecc\nRowYkW973bp1dHR04O60trZy9913x39QIlLT6vKW1KVLgxzChg3BGcK8edV9uhcH3ZIqEo8rrgjy\nCHv3QmNj8EfozTen3avot6TW3ZkCVP81PxHJpiuugMIT/717981nITBEUXc5BRGRpCxeXNnyLFJQ\nEBGJyd69lS3PIgUFEZGYNDZWtjyLFBRERCKIUh6nq6v07/a3PIvqMtEsIlKJXEns3K3suZLY0Pum\nlVwyOYt3H0WlM4WYmBnXXHNNfv6b3/wmc+fOjX0/KqktMvwqKY9z882wZ09Q+WjPnuoKCFCvQaF4\nbEYMYzX2228/7rzzTjZv3jzktgZSHBQeffTRRPcnIsmVx8mi+gsKc+fC1VfvCwTuwfwQ/6pvamqi\nq6uL+fPn91m3adMmPvrRj3LCCSdwwgkn8Mgjj+SXv+9976Ojo4NLL72Utra2fFD50Ic+xJQpUzju\nuONYHN7PppLaIumoiZLYUUUppTrYCTgQWAY8A6wFTilaPw3YBqwOpxvKtTmk0tk9Pe6zZwf1bGfP\nLj0/SKNHj/Zt27Z5W1ubb9261b/xjW/4nDlz3N39wgsv9BUrVri7+/r16/2YY45xd/dPf/rT/tWv\nftXd3ZcvX+6Ab9q0yd3dt2zZ4u7uO3bs8OOOO843b96c30/xft2HXlJbpbNF+pflkthREbF0dtKJ\n5gXAfe5+npmNAFpKbLPC3T+QcD8CZpD7S37BgmACmD07WD7EB9gfcMABXHTRRSxcuJBRo0bllz/4\n4IP5x2wCvPrqq7z22ms8/PDD3HXXXQDMmDGDgw46KL/NwoUL8+teeOEFnn32WQ455JB+93322Wdz\n5ZVXsmvXLu67775eJbWfeuopli1bBsC2bdt49tlnOfLII4d0rCL1JJdMrofyOIkFBTM7ADgdmAXg\n7n8F/prU/iLLBYZcQIBYAkLOVVddRUdHBxdffHF+WU9PDytXruwVKKD/0toPPfQQDz74ICtXrqSl\npYVp06axc+fOAferktoiyaqX8jhJ5hSOAjYB3zWzJ83sdjMbXWK7U8xsjZktN7PjSjVkZl1m1m1m\n3Zs2bRpar3I5hEKFOYYhOvjggzn//PP5zne+k1925plnctNNN+Xnc89deM973sMPf/hDAB544AFe\neeUVIPhr/qCDDqKlpYVnnnmGxx57LP+7KqktIklKMig0AR3AInc/HtgOfKFomyeANnefDNwIlKz1\n7O6L3b3T3TtbW1sH36NcQFiwILhk1NMT/FywINbAcM011/S6C2nhwoV0d3czadIkjj32WG655RYA\n5syZwwMPPEBHRwfLly/nsMMOY8yYMcyYMYM9e/YwadIkvvSlL3HyySfn28qV1J5Z4k+WM888k1/8\n4hecccYZvUpqH3vssXR0dDBx4kQuvfRS9uzZE8txikgNipJ4GMwEvBlYVzB/GvD/yvzOOuDQgbYZ\n6jOafc6c3knlXLI5TAoPp507d/ru3bvd3f3RRx/1yZMnD3sfcpRoFsmmJUvc29rczYKfg01uk3ai\n2d3/ZGYvmNk73P13wHTg6cJtzOzNwJ/d3c3sRIIzly1J9QkIbj1135dDyOUYYsopVGLDhg2cf/75\n9PT0MGLECG677bZh74OIZFfUkdRxSvruo88CS8M7j54HLjazywDc/RbgPOByM9sDvAFcEEa0ZBUH\ngBQCAsDRRx/Nk08+mcq+RST7BhpJXZVBwd1XA8VP+rmlYP1NwE3EwN2xlL7cq91wxGERqVwaI6lr\nYkTzyJEj2bJli77cBsHd2bJlCyNHjky7KyJSJI2R1DVRJXXcuHFs3LiRId+uWqdGjhzJuHHj0u6G\nSCT19Iz1efN65xQAWlqC5UmpiaDQ3NysEboidSCNxGua0hhJbdV2yaWzs9O7u7vT7oaIpGDChCAQ\nFGtrg3Xrhrs31cXMVrl7cY63j5rIKYhIfainEtZpUVAQkapRVyWsU6KgICJVY968INFaKOnEa71R\nUBCRqjFzZvD847a2YMxpW1swX4tJ5rTUxN1HIlI/6qWEdVp0piAiInkKCiIikqegICKZsHRpMA6h\noSH4uXRpttrL+n7jopyCiKQu7pHKaY18roUR1xrRLCKpi3ukclojn7M84lojmkWkasQ9Ujmtkc+1\nMOJaQUFEUhf3SOW0Rj7XwojrRIOCmR1oZsvM7BkzW2tmpxStNzNbaGbPmdlTZtaRZH9EBqPaE4fV\nYN48aG7uvay5efAjldMa+VwLI66TPlNYANzn7scAk4G1RevPBo4Opy5gUcL9EalILnG4fn3waO9c\n4lCBIX5xPiU3rZHPtTDiOrFEs5kdAKwBjurvuctmdivwkLv/IJz/HTDN3V/qr10lmmU4ZTlxWEv0\nPicvC4nmo4BNwHfN7Ekzu93MRhdtczjwQsH8xnBZL2bWZWbdZtatp6vJcKqFxGE10PucHUkGhSag\nA1jk7scD24EvFG1T6gSxz1mFuy92905372xtbY2/pyL9qIXEYTXQ+5wdSQaFjcBGd388nF9GECSK\ntzmiYH4c8GKCfRKpSC0kDnOyPMK3lt7nqufuiU3ACuAd4eu5wDeK1r8fWE5wxnAy8MtybU6ZMsVF\nhtOSJe5tbe5mwc8lS9LuUeWWLHFvaXEP0uXB1NKS/LFUst9aeJ+zDOj2CN/biY5oNrN24HZgBPA8\ncDHw8TAY3WJmBtwEzAB2ABe7+4BZZCWaRSqnEb4SNdGsMhcidaChIfg7vZgZ9PTU3n6lryzcfSQi\nGaERvhKVgoLIMEujRHQ1jPDVyPGMiJJ4yNKkRLNUs7gTvtWQyI2y37QS4fWELCSak6CcglSzWikR\nHbdaOY4sU05BJINqpUR03GrlOGqBgoLIMKqVEtFxq5XjqAUKCiLDKO6E77x5QWK2UEPD0BLIaSR8\nNaI5OxQURIZR3KWVH3mk7/3+PT3B8sFIq1R4LZScrhVKNItUsaYm2Lu37/LGRtizp/L2lPCtXUo0\ni9SBUgFhoOXlKOErCgoiVayxsbLl5SjhKwoKUnPiTpRecUVwmcYs+HnFFUPbLk5dXZUtL0cJX1FQ\nkJoSd6L0iitg0aJ9l2P27g3mi7/wo24Xt3e/u+9ZQWNjsHwwlPAVJZqlpsSdKI2ayI074RuVEsMS\nlRLNUpfiTpRGTeTGnfCNSolhiZuCgtSUuBOlURO5cSd8o1JiWOKWaFAws3Vm9mszW21mfa75mNk0\nM9sWrl9tZjck2R+pfXEnSqMmcitJ+MaZCFdpaoldlFKqg52AdcChA6yfBvy0kjZVOlvKibtE9OWX\nuzc2BuWcGxuD+VL7bGrqXfq5qanvvpMoEa3S1BIFWSidbWbrgE5339zP+mnAte7+gahtKtEsWRQ1\n4atnJUtaspJoduABM1tlZv3dOX2Kma0xs+VmdlypDcysy8y6zax706ZNyfVWZJCiJnzTSgwrIS1R\nJR0U3u3uHcDZwKfN7PSi9U8Abe4+GbgRuLtUI+6+2N073b2ztbU12R6LDELUhK+elSxZl2hQcPcX\nw58vA3cBJxatf9XdXw9f3ws0m9mhSfZJJCeNhK+elSyZFyXxMJgJGA2MKXj9KDCjaJs3s28A3YnA\nhtx8f5MSzRKHtBK+lWwXNyWk6xtpJ5rN7CiCswOAJuBf3X2emV0WBqNbzOwzwOXAHuAN4HPu/uhA\n7SrRLHFQ4rU0vS+1K2qiWWUupC41NAR/Bxcz6/vQmnqi96V2ZeXuI5FMUuK1NL0v0m9QMLN3mdlj\nZvaCmS02s4MK1v1yeLonkowkEr5plM6Om0pny0BnCouAucC7gN8DD5vZW8N1zQn3SyRRcZeITqt0\ndtxUOlv6zSmY2Wp3by+Y/xtgMfA/gJs9GH8w7JRTkCxKq3S2SFRRcwpNA7dhY919G4C7/4eZfRT4\nMXBwTP0UqQlplc4WidtAl4++DryzcIG7PwVMB+5MslMi1Sat0tkices3KLj7v7r7YyWWb3D3v0u2\nWyLVJa3S2Um0J/VtoMtHIhLRzTcHPxcvDi4ZNTYGASG3PCf3DOkdO4L53DOkYXDJ3LjbE9HgNZFh\nFPeIYY1Alqg0eE0kg+IuYa2S2BK3skHBzN5uZj8zs9+E85PM7H8m3zWRIsVntVV2lgvxjxjWCGSJ\nW5QzhduA64HdkL8D6YIkOyW14YwzggFQuemMM0pvFylROncuXH31vkDgHszPnZtI3yvuX0Tz5sGI\nEb2XjRgx+BHDSYxAVuK6zpUrowr8Kvz5ZMGy1VFKsCYxqXR2dZg+vXf55dw0fXrv7SKVau7pcZ89\nO1g5e3bp+YTEXUp6yRL35ube7TU3D0/J7qhtqXR2bSJi6ewoQWE58FbgiXD+PGB5lMaTmBQUqkOp\ngJCbCrW1ld6mra2owcJAkJsSDggV9S+l9uKW9f7J4EUNCmXvPgqfi7AYOBV4BfgjMNPdS9zzkDzd\nfVQdzPpfV/hPrqJSze7BL+T09Ay8oxjEXUo666Wps94/GbxY7j4yswag093PAFqBY9z9PWkFBKk9\nkROlHuYQChXmGBJSFYnh4vdgCO+JEtcyYFBw9x7gM+Hr7e7+WiWNm9k6M/u1ma02sz5/3ltgoZk9\nZ2ZPmVkqRfakMlESkdOnl/7d4uWREqW5gLBgAf88ZjYN9PDPY2bDggWJB4Z586C5qCZwc3OGEsMx\nJ+BVOlui5BS+BFwLHEFQCO9g4OAo16aAdcChA6z/W4KchQEnA4+Xa1M5hXRVkogsTjYXJ5kL2yyX\nKF3z4Tl+U9Nsh56wvR6/qWm2r/nwnBiPrnTfRozofRwjRmQkMZxQAj6tZ0hLsogxp/DH0rHEjyoX\ncMxsHcHlp839rL8VeMjdfxDO/w6Y5u4v9demcgrpSmsEbbBfJ/j7Icdpa7Nh2G/f5ZkZMVxwFpU3\nezbMn594vkWqSyae0RwGlFcAB25198VF638K/KO7PxzO/wz4vLt3F23XBXQBjB8/fsr6Uv9LZVik\nlYist/1WJIUEvFSf2MpcmNlFpaaI/Xi3Bw/jORv4tJmdXtx8id/p81/Q3Re7e6e7d7a2tkbctSQh\nrURkuvst/ifp2Um8JpGAjzFxLdUnyojmEwqm0wge0XlOlMbd/cXw58vAXcCJRZtsJMhV5IwDXozS\ntsQvSgI5rURkJfuNc0TuPR1zWdhwNfsCg7Ow4Wru6Zg7+EbjUnjpaPbs4Axh9hAT8CmOHJeMiJJ4\nKJyAscA9EbYbDYwpeP0oMKNom/fTO9H8y3LtKtGcjEoSyGklIqPsN9YRuT09vvasIHE7nyDJPZ9g\nfu1ZyQ+ci2TOnN5J5Vyyec6cyttKceS4JI+4RjT3+QVoBtZG2O4oYE04/Rb4Yrj8MuCy8LUB/wT8\nAfg1QVJaQSEFtTKSNfYRyOP3BYLcNJ/Z3jY+Q1+QxV/WQ/nyTmnkuCQvalCIcvfRT9h37twAHAv8\nyN0/P4QTlEHT3UfJqIqEagTJjEB2vOBKq9GDmVXV+1IRJa5rUpzPU/gm8H/C6WvA6WkFBElOqiNZ\nY0xsxj4C+QhnPr0TufO5mvFHVGHyNcr7nETiWqpKlKDwt+7+n+H0iLtvNLOvJ94zGVapjWStILEZ\ndyK8bHvu3PfOq7mKBXyb2Rg9fJvZXMUC7ntnlX1RRnmfc8viTFxL9Sl3fYmwOmrRsqeiXJtKYlJO\nITnDnkCuILEZdyI8cntz5vjas4IcglmQY1h71iATuWmpJIEcZ+JaMoWh5hTM7HLgCoKE8R8KVo0B\nHnH3TyQarfqhnEKNKfzrNKfEiNxUn23s3vuaevF8NYj4Pue3rfbjlT6GPKLZzMYCBxHkEb5QsOo1\nd/9LLL0cBAWFGhQhsVlvJawToQRyXRtyotndt7n7One/0INS2W8Q3IW0v5llZTynVLuIic2KEsjF\n3+olvuUrai/GRHgi0kogZ/19kUGJUubig2b2LMHDdf6ToPLp8oT7JfWggsRm5ATytGkwZcq+QNDT\nE8xPm9Zrs8jtZX2Eb1oJ5Ky/LzJoUe4++l8Eo41/7+5HAtOBRxLtldQHMzjwwN7XtufPD+YPPLDX\npY2ZM2Hx4uCav1nwc/HiYHleTw9s2warV+8LDFOmBPPbtvU6Y4jUnjts3dr7yzP35bp1a/p/GUft\nXwXvc6z7lepULhNNmLEmGJncEL4uW44iqUl3H9WgOEfk7t3r3t7e+7ai9vZg+WD7luURvpX0TyOf\n6xoxjmh+EPgQ8I/AIcDLwAnufmpyoap/SjRLWT090Ni4b37v3t4J1kplPUGbVv+y/r5IL3GOaD4X\n2AFcBdxHcHvqB4fWPZGE9PRAR9FTXTs6Bn9LUZoJ2rQSyFGktV9JXpTTCaANOCN83UJY/TSNSZeP\npF9797q/6U3BpYzJk4P5yZOD+Te9qfJLSElUDY06OCzKdmlVNVU11apExMtHTeWChpn9HcFTzw4G\n3gocDtxCkHAWyQ4zaAr/SU+dGsxPnQpr1gTLK7200V+CFoaeoIWgrcK7gjwcJBZ1u7j7F1Va+5Xh\nUS5qAKuBEcCTBct+HSXiJDHpTEEG1NPjfuWV+5KfEMwPNak60HylbUVJ0KaVQK5EWvuVQSHGRPPj\n7n6SmT3p7sebWRNBPaRJCcerkpRolrI84wnQqP3L+nFIVYkz0fyfZvb3wCgzex/wI+AnQ+2gDFFx\nMI8jwRe1zST2HUWEkcoVJUDjPo4o7UXtXyXHIRKncqcSBIHj7wiCwbLwtUU5DQl/vxF4EvhpiXWz\ngE0El6hWA5eUa2+ol4/SepRkrJKoZBlnAjQJU6f2Hm+QG48wdeq+bdKsBhpnYliJXEkAES8f9Xum\nkKtv5O497n6bu3/M3c8LX1fy58psYO0A6+9w9/Zwur2Cdiu2dCl0dQXVMd2Dn11dQ3uw+7DzBEaT\nRm0ziX1HEXWkctSRu3EfR9T2ovYv7hHIIpXoL1pQ8BwF4MdRIkyJNsYBPwPeS/9nCjdV0uZQzhRq\n5TnEiYwmTSIBGqdKRipHSYDGfRxJJIaVyJUYEcPzFJ509+OLX1fCzJYRlN4eA1zr7h8oWj8rXL8J\n+D1wtbu/UKKdLoLbYhk/fvyU9aUK4UdQU+WSPYEkZNQ2k9h3FFkfqZzW+yISQRyJZu/nddQOfAB4\n2d1XDbDZT4AJHtzJ9CDwvZIdcV/s7p3u3tna2lppV/JSfQ5xnHKXJwoNNZkatc1K9h2n3CWjQoXV\nUCsV93uYxPtSyWcXZTuRKPo7hQD2Aq8CrwF7wte5+VfLnYIQnAFsJCi1/SeCUhlLBti+EdhWrt2h\nXD6q5JGOmZVEMjXrCdDCS0e5S0bF85WI+z3M+shnEY9++ajfEc3u3tjfuojB5nrgegAzm0Zw+ajX\nIzzN7DB3fymcPYeBE9JDliuL/MUvwoYNwRnCvHlF5ZKzLupoUvdoo2IraTOtkawNDTB2LLS3w6pV\nwfyqVcGZwtixlV9Civs9zPrIZ5FKRIkcQ52AaYSJZuArwDm+72zitwRluf8DOKZcWxrRHEoimZr1\nBGjxGcFgy2HnxP0eZn3ks9Q14hrRnDUa0VwhV/JzyNJ6D6PuV5+xRBDniGapVp5SUrhaREnQVvIe\nRmmvkr5lOfEvtSvK6USWJl0+ikijYgcWdwI5zoRv1hP/UpWIq3S2VCmVN+6fx5xAjtpeVFlP/EtN\nU06h1hV/IVX6BVWrcpddcl/k0PvLtXjbcu9hJe1V0scon50+Y4kgak5BQUHqV9wJWiV8JcOUaJbK\nRU2URilhncR+4xR3glYJX6kRCgoSmDu395dY7ktu7tze202b1ru8RK78xLRpye43ToWXembPDo5h\n9uzeVU7TbE8kRQoK0jtROlDp56glrOPeb9ziLk2tUtdSQ5RTkEDURGlhIMgpLD+R1H6TEHeCVglf\nyTAlmqVyUROlWS9hLSJ9KNEN0AAPAAANSUlEQVQslXGHq67qveyqq/pewunpgY6O3ss6Ovp/XvJA\n87llStCKZIaCggRfwKecAgsXwpVXBl/wV14ZzJ9ySu+cwlveAmvWwOTJwRnC5MnB/Fve0jswREkg\nK0ErkjkKChKdGTSFg+CnTg3mp04N5puaKn8GshK0ItkTpRZGlibVPkpIT4/7lVd6rxLMV15ZulRz\n1O3SKDktIiWh0tlSsagJ37i3E5HEKdEsgaijj6MmfCvZLkriuhJRRz6nMUJapEYkHhTMrNHMnjSz\nn5ZYt5+Z3WFmz5nZ42Y2Ien+1JWoo4+jJnwr2S5K4roSUUc+pzFCWqSGDMeZwmz6f/byp4BX3P1t\nwHzg68PQn/pQyejjqAnftBLDURPXaY2QFqklURIPg52AccDPgPcSPqO5aP39wCnh6yZgM+GAuv4m\nJZorsHeve3t772Rve3v/zzaO8xnNURPSUemZxSJDQhYSzWa2DPgaMAa41t0/ULT+N8AMd98Yzv8B\nOMndNxdt1wV0AYwfP37K+vXrE+tzzYl79HEl4k40K8EtMmipJ5rN7APAy+6+aqDNSizrE6XcfbG7\nd7p7Z2tra2x9HFZpJD9zl4wKFeYYisXZx7gTzXEnwkWktCinE4OZCM4QNgLrgD8BO4AlRdvUx+Wj\nOJ/fG1XhpaPcJaPi+aT62NPjftJJvS8Z5S4lnXRS5Zdy9MxikSEj4uWjxM4U3P16dx/n7hOAC4Cf\nu/snija7B/hk+Pq8cJva+pMureRnQwOMHdu7gumqVcH82LG9L6+k1ceosp4IF6klUSLHUCdgGmGi\nGfgKcE74eiTwI+A54JfAUeXaqsozhTSTn8VnBAMlmePsY9yJ5lybA81Xup1IHSELieYkVO2I5mpI\nfqaVGBaRxKWeaJYCaSY/oyaP4+6jEr4iVUlBIWmF1+eHuzx01NG9cfcxzWMWkSFpSrsDNa+/5CcM\n3yhgCPZZ+EXtXj5BO9g+pnXMIjJkyikMl8Iv4VLzSe2zkucfx93HNI5ZRErSM5oloGSviKBEs0Ay\n5atFpKYpKNQqT6B8tYjUPAUFERHJU1CoVWawcuW+s4OGhn1nDStXKq8gIiUp0VzrlGgWEZRoFqh8\nVHHU0c8iUrMUFGpVpaOK9WxjEUEjmmtXJaOKKxn9LCI1TTmFWhd1VHGlo59FpKpoRLNUTklpkZql\nRPNQ1VvStZKkdL29NyJ1JLGgYGYjzeyXZrbGzH5rZl8usc0sM9tkZqvD6ZKk+lOReku6VpKUrrf3\nRqTOJHmmsAt4r7tPBtqBGWZ2cont7nD39nC6PcH+RJP15xUnIeqzjevxvRGpM8OSUzCzFuBh4HJ3\nf7xg+Syg090/E7WtYckp1GvSNUpSul7fG5Eql4lEs5k1AquAtwH/5O6fL1o/C/gasAn4PXC1u79Q\nop0uoAtg/PjxU9avX59Yn/OUdO2f3huRqpOJRLO773X3dmAccKKZTSza5CfABHefBDwIfK+fdha7\ne6e7d7a2tibZ5dwO9Xzh/ui9Ealpw3L3kbtvBR4CZhQt3+Luu8LZ24Apw9GfAen5wv3TeyNS8xIb\n0WxmrcBud99qZqOAM4CvF21zmLu/FM6eA6xNqj+R6fnC/dN7I1LzEsspmNkkgstBjQRnJD9096+Y\n2VeAbne/x8y+RhAM9gB/IUhEPzNQu8M2eE3PF+6f3huRqpOJRHMSNKJZRKRymUg0Z1LU0bgatSsi\ndai+gkLU0bgatSsidap+gkLU0bgatSsiday+cgpRR+Nq1K6I1BglmvsTdTSuRu2KSA1RormUqKNx\nNWpXROpU/QSFqKNxNWpXROpY/TyjOepoXI3aFZE6Vp85hajPLNaoXRGpEcop9Kf4i72/L/qo24mI\n1JD6CwoiItIvBQUREclTUBARkTwFBRERyVNQEBGRPAWFoVKJbRGpIYkFBTMbaWa/NLM1ZvZbM/ty\niW32M7M7zOw5M3vczCYk1Z9EqMS2iNSYJM8UdgHvdffJQDsww8xOLtrmU8Ar7v42YD5Fz3DONJXY\nFpEalFiZCw+GSr8ezjaHU/E35bnA3PD1MuAmMzOvhmHWheUvFizYV2ZbJbZFpIolmlMws0YzWw28\nDPy7uz9etMnhwAsA7r4H2AYcUqKdLjPrNrPuTZs2JdnlyhQGhhwFBBGpYokGBXff6+7twDjgRDOb\nWLRJqW/PPmcJ7r7Y3TvdvbO1tTWJrg6OSmyLSI0ZlruP3H0r8BAwo2jVRuAIADNrAsYCfxmOPg2Z\nSmyLSA1KLKdgZq3AbnffamajgDPom0i+B/gksBI4D/h5VeQTQCW2RaQmJVY628wmAd8DGgnOSH7o\n7l8xs68A3e5+j5mNBP4FOJ7gDOECd39+oHaHXDo7biqxLSJVIGrp7CTvPnqK4Mu+ePkNBa93Ah9L\nqg/DQiW2RaSGaESziIjkKSiIiEiegoKIiOQpKIiISJ6CgoiI5CkoiIhIXmLjFJJiZpuA9cO4y0OB\nzcO4v6TUynFA7RyLjiNbav042ty9bJ2gqgsKw83MuqMM+Mi6WjkOqJ1j0XFki44joMtHIiKSp6Ag\nIiJ5CgrlLU67AzGpleOA2jkWHUe26DhQTkFERAroTEFERPIUFEREJE9BIWRm68zs12a22sz6PLDB\nAgvN7Dkze8rMOtLoZzkRjmOamW0L1682sxtKtZM2MzvQzJaZ2TNmttbMTilaXy2fR7njqJbP4x0F\nfVxtZq+a2VVF22T+M4l4HNXymVxtZr81s9+Y2Q/C59MUrt/PzO4IP4/HzWxCpIbdXVOQV1kHHDrA\n+r8FlhM8V/pk4PG0+zzI45gG/DTtfkY4ju8Bl4SvRwAHVunnUe44quLzKOpzI/AngsFQVfeZRDiO\nzH8mwOHAH4FR4fwPgVlF21wB3BK+vgC4I0rbOlOI7lzg+x54DDjQzA5Lu1O1yMwOAE4HvgPg7n/1\n4DnfhTL/eUQ8jmo0HfiDuxdXFsj8Z1Kkv+OoFk3AqPD59i3Ai0XrzyX4owRgGTDdrPxTwBQU9nHg\nATNbZWZdJdYfDrxQML8xXJY15Y4D4BQzW2Nmy83suOHsXERHAZuA75rZk2Z2u5mNLtqmGj6PKMcB\n2f88il0A/KDE8mr4TAr1dxyQ8c/E3f8L+CawAXgJ2ObuDxRtlv883H0PsA04pFzbCgr7vNvdO4Cz\ngU+b2elF60tF2Czez1vuOJ4gOF2eDNwI3D3cHYygCegAFrn78cB24AtF21TD5xHlOKrh88gzsxHA\nOcCPSq0usSxrnwlQ9jgy/5mY2UEEZwJHAm8BRpvZJ4o3K/GrZT8PBYWQu78Y/nwZuAs4sWiTjcAR\nBfPj6Hu6lrpyx+Hur7r76+Hre4FmMzt02Ds6sI3ARnd/PJxfRvDlWrxN1j+PssdRJZ9HobOBJ9z9\nzyXWVcNnktPvcVTJZ3IG8Ed33+Tuu4E7gVOLtsl/HuElprHAX8o1rKAAmNloMxuTew2cCfymaLN7\ngIvCOyxOJjhde2mYuzqgKMdhZm/OXVc0sxMJ/g1sGe6+DsTd/wS8YGbvCBdNB54u2izzn0eU46iG\nz6PIhfR/ySXzn0mBfo+jSj6TDcDJZtYS9nU6sLZom3uAT4avzwN+7mHWeSBNsXazer0JuCv8d9AE\n/Ku732dmlwG4+y3AvQR3VzwH7AAuTqmvA4lyHOcBl5vZHuAN4IIo/1BS8FlgaXia/zxwcRV+HlD+\nOKrl88DMWoD3AZcWLKu6zyTCcWT+M3H3x81sGcGlrj3Ak8BiM/sK0O3u9xDc4PAvZvYcwRnCBVHa\nVpkLERHJ0+UjERHJU1AQEZE8BQUREclTUBARkTwFBRERyVNQkLpjZnuLKmVOGEQbB5rZFfH3Lt/+\nMWa20sx2mdm1Se1HpJhuSZW6Y2avu/v+Q2xjAkElzYkV/l6ju++NsN1/A9qADwGvuPs3B9NPkUrp\nTEGE4MvazL5hZr+y4FkAl4bL9zezn5nZExY8p+Lc8Ff+EXhreKbxDQtq8P+0oL2bzGxW+Hqdmd1g\nZg8DHzOzt5rZfWHRwhVmdkxxf9z9ZXf/FbA78YMXKaARzVKPRpnZ6vD1H939w8CnCMoynGBm+wGP\nmNkDBFUmP+zur4b1bx4zs3sICttNdPd2CB7MUmafO939PeG2PwMuc/dnzewk4GbgvXEfpMhgKChI\nPXoj92Ve4ExgkpmdF86PBY4mKCr21bDabA9BOeI3DWKfd0Bw5kFQuOxHtq+0/X6DaE8kEQoKIgED\nPuvu9/daGFwCagWmuPtuM1sHjOz76+yh9+XY4m22hz8bgK0lgpJIJiinIBK4n6AIWjOAmb09rDQ7\nFng5DAh/Q5D8BXgNGFPw++uBYy14Lu5YgqqVfbj7q8Afzexj4X7MzCYnc0gildOZgkjgdmAC8ERY\ningTwZ0/S4GfmFk3sBp4BsDdt5jZI2b2G2C5u19nZj8EngKeJaha2Z+ZwCIz+59AM/BvwJrCDczs\nzUA3cADQY8HD5Y8Ng4pIYnRLqoiI5OnykYiI5CkoiIhInoKCiIjkKSiIiEiegoKIiOQpKIiISJ6C\ngoiI5P1/87q0r0vRhmUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb5bd320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Original Data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很明显，从图中可以看出，正类和负类样本并不是线性可分的。这时候，我们就需要对PCA进行改进，寻找合适的分类线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征归一化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2YFfV99/H3dx8QVxCV0KQJshir\nUUJgXTBRG4UGg5gHE5NoYonWNBYraVi81Kq1CbQp3u1lGgKxPpDHuzfb1MSHNDGilKZWNJgKilaD\nEZOIcpu7QSpIQAjs+d5/zJzl7OHs7pw9Mzsz53xe1zXX7sw5Z+Y3I57vznzm9xtzd0RERJrSboCI\niGSDCoKIiAAqCCIiElJBEBERQAVBRERCKggiIgKoIIgAYGaLzWxl+PsEM/uNmTXHvI0zzexnca5z\ngG29YGZnD8e2pH6oIMiwCL+g/tvMjihZdpmZPZhisypy9xfdfZS799SyHjNzM/u9kvWudfe31d7C\neJW3UxqXCoIMpxagq9aVWED/dkVipv+pZDjdBFxtZkdVetHMzjCzx8xsZ/jzjJLXHjSzJWb2CLAH\neGu47G/M7MfhJZ4fmNlYM+s2s9fCdUwsWccyM3spfG2DmZ3ZTzsmhn81t5jZ6eG6i9NeM3shfN87\nzWydme0ws1+Z2c1mNiJ87aFwdU+Gn/u4mc00s60l2zk53IcdZvaMmZ1X8tq3zOwfzOyHZrbLzH5i\nZsf3d2DN7GIz22Jm283shrLXqm3n0WZ2r5ltM7NXw9/H97dtqR8qCDKc1gMPAleXv2BmxwA/BJYD\nY4EvAT80s7Elb7sYmAeMBraEyz4RLn8LcDywDvgmcAywCVhU8vnHgI7wtX8CvmtmIwdqsLuvCy8f\njQKOBh4Fvh2+3ANcCbwBOB2YBcwPP3dW+J6p4efvKNvfVuAHwGrgd4DPAt1mVnpJ6SLgr8LtPg8s\nqdRGM5sE3BoehzcTHL/SL/Bq29lEcAzbgQnA68DNAx0nqQ8qCDLcPg981szGlS1/P7DZ3f+Pux9w\n928DzwIfLHnPt9z9mfD1/eGyb7r7z919J7AK+Lm7r3H3A8B3gVOKH3b3le6+Pfz83wOHAdVc018O\n7AZuCNe3wd0fDdf3AnA7MCPiuk4DRgF/6+6/dfcfAfcSFIGiu939P8N96SYoZpV8DLjX3R9y933A\n54BC8cVq2xkeo7vcfY+77yIoRFH3S3KsJe0GSGNx96fN7F7gOoK/4IvezMG/+ou2EPzlX/RShVX+\nd8nvr1eYH1WcMbOrgMvCbTlwJMFfzYMys8uBmcBp7l4Il51IcCYzHWgj+P9pQ5T1hW14qbiuUPn+\n/r+S3/eU7kuldRVn3H23mW0vaXtV7TSzNmApMIfg7ARgtJk11xq0S7bpDEHSsAj4E/p++b1McImi\n1ATg/5bMD3lo3jAvuBa4EDja3Y8CdgIW8bNfAD4UnokU3UpwFnOCux8J/EWU9YVeBo4tC8fL9zeq\nXwHHlrS3jeCy0VDbeRXBmdO7wvcXLytF3TfJKRUEGXbu/jxwB7CgZPF9wIlm9odhmPtxYBLBZZQ4\njAYOANuAFjP7PMEZwoDM7NiwrZe4+3MV1vka8BszOwm4ouz1/wbe2s+qf0Jw+enPzazVzGYSXB77\n52i708edwAfM7N1hWPzX9P1/u9p2jiY4u9oRZjuLkIaggiBp+Wugt0+Cu28HPkDw1+l24M+BD7j7\nKzFt7wGCjOE5gksze6l8CarcLOBNwJ0ldxo9E752NfCHwC7gqwSFo9Ri4H+Hd/dcWPqCu/8WOA84\nF3gFuIWg6Dxb7Y65+zPAZwiC8l8BrwJbS95SbTu/DBwetutR4P5q2yT5ZHpAjoiIgM4QREQkpIIg\nIiKACoKIiIRUEEREBMhZx7Q3vOENPnHixLSbISKSKxs2bHjF3ctHBzhErgrCxIkTWb9+fdrNEBHJ\nFTMrHwWgIl0yEhERQAVBRERCKggiIgLkLEOoZP/+/WzdupW9e/em3ZRcGjlyJOPHj6e1tTXtpohI\nynJfELZu3cro0aOZOHEiZhqMsRruzvbt29m6dSvHHXdc2s0RkZTl/pLR3r17GTt2rIrBEJgZY8eO\n1dmViAB1UBAAFYMa6NiJxKe7GyZOhKam4Gd3d9otqk7uLxmJiGRBdzfMmwd79gTzW7YE8wBz56bX\nrmrUxRlC2pqbm+no6GDy5MlccMEF7Cn+i6jCZZddxk9/+lMAbrzxxj6vnXHGGbG0U0SSc8MNB4tB\n0Z49wfK8UEGIweGHH87GjRt5+umnGTFiBLfddlvV6/ja177GpEmTgEMLwo9//ONY2ikiyXnxxeqW\nZ1HDFYSkr/GdeeaZPP/88wB86UtfYvLkyUyePJkvf/nLAOzevZv3v//9TJ06lcmTJ3PHHcHDq2bO\nnMn69eu57rrreP311+no6GBueJ45alTwbPWPf/zj3Hfffb3buvTSS7nrrrvo6enhmmuu4dRTT2XK\nlCncfvvt8e6UiAxqwoTqlmeSu+dmmjZtmpf76U9/esiy/qxc6d7W5g4Hp7a2YHktjjjiCHd3379/\nv5933nl+yy23+Pr1633y5Mn+m9/8xnft2uWTJk3yxx9/3O+8806/7LLLej+7Y8cOd3efMWOGP/bY\nY33WV77+u+++2y+55BJ3d9+3b5+PHz/e9+zZ47fffrt/4QtfcHf3vXv3+rRp0/wXv/hF5PZXcwxF\npLKkvl/iAKz3CN+xDXWGkNQ1vuJf9NOnT2fChAl8+tOf5uGHH+b888/niCOOYNSoUXzkIx9h7dq1\nvOMd72DNmjVce+21rF27ljFjxkTezrnnnsuPfvQj9u3bx6pVqzjrrLM4/PDDWb16Nf/4j/9IR0cH\n73rXu9i+fTubN2+ubadEpCpz58KKFdDeDmbBzxUr8hMoQ4PdZZTUNb5ihlDK+3lW9YknnsiGDRu4\n7777uP7665k9ezaf//znI21n5MiRzJw5kwceeIA77riDiy66qHdbX/nKVzjnnHNq2xERqcncufkq\nAOUa6gxhOK/xnXXWWXzve99jz5497N69m3vuuYczzzyTl19+mba2Nj75yU9y9dVX8/jjjx/y2dbW\nVvbv319xvZ/4xCf45je/ydq1a3sLwDnnnMOtt97a+5nnnnuO3bt3x79TIlLXGuoMYcmSvvcJA7S1\nBcvj1tnZyaWXXso73/lOILit9JRTTuGBBx7gmmuuoampidbWVm699dZDPjtv3jymTJlCZ2cn3WWp\n9+zZs7nkkks477zzGDFiRO+6X3jhBTo7O3F3xo0bx/e+9734d0pE6pr1d2kji6ZPn+7lD8jZtGkT\nJ598cuR1dHcHmcGLLwZnBkuW5PsULw7VHkMRqWz+/CA36OmB5ubgD9Bbbkm7VWBmG9x9+mDva6gz\nBMj/NT4Ryab586H0hL+n5+B8FopCFA2VIYiIJGXFiuqWZ5EKgohIDHp6qlueRSoIIiIxaG6ubnkW\nqSCIiAwiypA3xZFNoy7PooYLlUVEqhF1WOticJzFu4yiSu0MwcyONbN/N7NNZvaMmXWl1ZZamRlX\nXXVV7/wXv/hFFi9eHPt2NCy2yPCrZsibW26BAweCkYwOHMhXMYB0LxkdAK5y95OB04DPmNmkxLda\n3u8ihn4Yhx12GHfffTevvPJKzesaiIbFFhl+9TCsdVSpFQR3/5W7Px7+vgvYBLwl0Y0uXgxXXnmw\nCLgH8zX+Nd/S0sK8efNYunTpIa9t27aNj370o5x66qmceuqpPPLII73L3/ve99LZ2cnll19Oe3t7\nb0H58Ic/zLRp03j729/OivCeNQ2LLZKOuhjWOqooQ6ImPQETgReBIyu8Ng9YD6yfMGHCIcO6Rh66\nuVBw7+oKxqTt6qo8P0RHHHGE79y509vb233Hjh1+0003+aJFi9zd/aKLLvK1a9e6u/uWLVv8pJNO\ncnf3z3zmM37jjTe6u/uqVasc8G3btrm7+/bt293dfc+ePf72t7/dX3nlld7tlG/XvfZhsTX8tUj/\nsjysdVREHP469VDZzEYBdwEL3f218tfdfQWwAoKhK2rYEBT/gl+2LJgAurqC5TU+bP7II4/kkksu\nYfny5Rx++OG9y9esWdP7aEyA1157jV27dvHwww9zzz33ADBnzhyOPvro3vcsX76897WXXnqJzZs3\nM3bs2H63fe6557JgwQL27dvH/fff32dY7Keeeoo777wTgJ07d7J582aOO+64mvZVpJEUg+NGGPIm\n1YJgZq0ExaDb3e8ehg0GX/7FYgCxFIOihQsX0tnZyac+9aneZYVCgXXr1vUpEtD/8NgPPvgga9as\nYd26dbS1tTFz5kz27t074HY1LLZIshplyJs07zIy4OvAJnf/0rBstJgZlCrNFGp0zDHHcOGFF/L1\nr3+9d9ns2bO5+eabe+eLz01497vfzXe+8x0AVq9ezauvvgoEf8UfffTRtLW18eyzz/Loo4/2flbD\nYotIktK8y+j3gYuB95jZxnB6X2JbKxaDZcuCy0SFQvBz2bJYi8JVV13V526j5cuXs379eqZMmcKk\nSZO47bbbAFi0aBGrV6+ms7OTVatW8bu/+7uMHj2aOXPmcODAAaZMmcLnPvc5TjvttN51FYfFnlvh\nT5XZs2fz0EMPcfbZZ/cZFnvSpEl0dnYyefJkLr/8cg4cOBDLfopI/Wms4a8XL4YdOw5eJioWiaOO\nqvlOo2rt27eP5uZmWlpaWLduHVdcccUhT10bLhr+WiSb4hquX8NfV7J4cVAEiplBMVOIKUOoxosv\nvsiFF15IoVBgxIgRfPWrXx32NohIdkXtIR2nxioIcOiXfwrFAOCEE07giSeeSGXbIpJ9A/WQTqog\n1MXgdnm67JU1OnYi2ZRGD+ncF4SRI0eyfft2fbENgbuzfft2Ro4cmXZTRKRMGj2kc3/JaPz48Wzd\nupVt27al3ZRcGjlyJOPHj0+7GSKRNNIz0Zcs6ZshALS1BcuTkvuC0Nraqp63Ig0gjZA1TWn0kM79\nbaci0hgmTgyKQLn2dnjhheFuTb5Eve009xmCiDSGRhqGOi0qCCKSCw01DHVKVBBEJBeWLAlC1VJJ\nh6yNRgVBRHJh7tzgecXt7UF/0vb2YL4eA+W05P4uIxFpHI0yDHVadIYgIiKACoKIiIRUEEQkdd3d\nQT+DpqbgZ3d3ttaX9e3GRRmCiKQq7h7IafVoroee1OqpLCKpirsHclo9mrPck1o9lUUkF+LugZxW\nj+Z66EmtgiAiqYq7B3JaPZrroSe1CoLIAPIeEubBkiXQ2tp3WWvr0Hsgp9WjuR56UqsgiPSjGBJu\n2RI8irsYEqooxC/OJ9um1aO5HnpSK1QW6UeWQ8J6ouOcPIXKIjWqh5AwD3Scs0MFQaQf9RAS5oGO\nc3aoIIj0ox5CwqIs99ytp+Oce+6em2natGkuMpxWrnRvb3c3C36uXJl2i6q3cqV7W5t7EI0HU1tb\n8vtSzXbr4ThnGbDeI3zHKlQWqXPquSsKlUUEUM9diU4FQaTOqeeuRKWCIDKM0hjmOQ89d9UjPCOi\nBA1ZmRQqS57FHe7mIbSNst20Qu9GgkJlkWypl2Ge41Yv+5FluQiVzewbZvZrM3s6zXaIDId6GeY5\nbvWyH/Ug7QzhW8CclNsgMizqZZjnuNXLftSDVAuCuz8E/E+abRAZLnGHu0uWBCFsqaam2sLiNMJd\n9VTOjrTPEEQaRtzDIz/yCBQKfZcVCsHyoUhruO96GDa6XqQeKpvZROBed5/cz+vzgHkAEyZMmLal\nUvok0oBaWqCn59Dlzc1w4ED161O4W79yESpH4e4r3H26u08fN25c2s0RyYxKxWCg5YNRuCuZLwgi\nUllzc3XLB6NwV9K+7fTbwDrgbWa21cw+nWZ7JN+SCETnzw8uzZgFP+fPr+19cZo3r7rlg1G4K6n3\nPq5mUk9l6U8SvV2vuKLv+orTFVcM7X1xW7nSvbm57zabm2vbZw1DXZ9QT2VpJEkEolFD27jD3agU\nAktUdRMqi0SRRCAaNbSNO9yNSiGwxE0FQepCEoFo1NA27nA3KoXAEjcVBKkLSQSiUUPbasLdOINv\nDS8tsYsSNGRlUqgsA0kiEL3iioPBbXNz5aB45Ur3lpa+4W5Ly6HbTyL41vDSEgUKlUWGR9RwV882\nlrQoVBYZJlHDXT3bWLJOBUGkRlHDXT3bWLJOBUEaTtwBa9RwV882lsyLEjRkZVKoLLVKKmCNGmjr\n2caSBhQqixxKAWv/dGzql0JlkQoUsPZPx0ZUEKShKGDtn46N9FsQzOwdZvaomb1kZivM7OiS1/5z\neJonEq+kgt00hr+Om4a/loHOEG4FFgPvAJ4DHjaz48PXWhNul0giknh+7/z5cOutBwez6+kJ5vNW\nFPRsY+k3VDazje7eUTL/B8AK4GLgFnfvHJ4mHqRQWbIoreGvRaKKGiq3DLwOG+PuOwHc/d/N7KPA\nXcAxMbVTJPfSGv5aJG4DXTL6O+Dk0gXu/hQwC7g7yUaJ5Elaw1+LxK3fguDu/+Tuj1ZY/qK7/0my\nzRLJj7SGv05ifdLYBrpkJCIR3HJL8HPFiuAyUXNzUAyKy4u6u4Ple/YE81u2HCwaQwlu416fiHoq\niwyTuHsCq2exRKWeyiIZE3dPYPUslrgNWhDM7EQz+zczezqcn2Jmf5l800RC5WexOTqrLRV3T2D1\nLJa4RTlD+CpwPbAfeu80+kSSjZJ8O/vsoGNTcTr77MrvixSILl4MV155sAi4B/OLFyfS9iG1MaIl\nS2DEiL7LRowYek/gJHoWK6RucIMNhwo8Fv58omTZxihDqcY9afjr7Js1q+/wycVp1qy+74s01HKh\n4N7VFbzY1VV5PkFxDwe9cqV7a2vf9bW2Jv9M5WrWpeGv6xMRh7+OUhBWAccDj4fzHwNWRVl53JMK\nQvZVKgbFqVR7e+X3tLeXrbC0CBSnYSgGVbUxpfXFLevtk6GLWhAGvcvIzN5KMGTFGcCrwC+Bue5e\n4f6GZOkuo+wz6/+10n9qTU2VowAzKBQqfLCp5OpmoTDwhmJSVRtTWF/cst4+GbpY7jIysyZguruf\nDYwDTnL3d6dRDKS+RA5EPcwMSpVmCgnKRQhcfhxqOC4KqWXAguDuBeDPwt93u/uuYWmVZFKUwHHW\nrMqfLV8eKRAtFoNly/jG6C6aKPCN0V2wbNmwFIUlS6C1bFzf1tYMhcAxB+4a/lqiZAifA64GjiUY\n1O4Y4Jgo16PinpQhpKeawLE8WC4PlEvXOVgg+uT5i/zmli6HQri+gt/c0uVPnr8oxr3rv30jRvTd\nlxEjMhICJxS4p/XMZ0kWMWYIv6xcR/yt8ZamwSlDSE9avWKD7TpQmhk47e2WeG/czPcELjmD6tXV\nBUuXDkvGIvkRNUPQ0BUSSVqBY5pBZy5C1pQCd8mX2IauMLNLKk0xNXKOmf3MzJ43s+viWKckI63A\nMc2gM9hGeUXw7ISsSQTuMYbUkj9ReiqfWjKdSfBYzfNq3bCZNQP/AJwLTAIuMrNJta5XqhclLE4r\ncKxmu3H3sv1+52KWN13JwaLgLG+6ku93Lq5txXEovVzU1RWcGXTVGLin3CtcMiBK0FA6AWOA71f7\nuQrrOR14oGT+euD6gT6jUDl+1YTFaQWOUbYbey/bQsE3nROEtEsJQu2lBPObzhmejnGDWrSob4Bc\nDJYXLap+XSn3CpdkEVdP5UM+AK3Apmo/V2E9HwO+VjJ/MXDzQJ9RQYhfvfROTWI/2iccLALFaSld\n3j4hQ1+O5V/UtXxxp9grXJIVtSBEucvoBxw8Z24iuLzzXXe/tpYzEzO7ADjH3S8L5y8G3ununy17\n3zxgHsCECROmbal024cMWS6C0wiS2I9gnY6XXFk1CphZro5NVVwhdT2K83kIXwT+Ppz+F3BWrcUg\ntJWgb0PReODl8je5+wp3n+7u08eNGxfDZqVUqr1Ty7/BB/njZCBJ7MeEY52l9A1tl3IlE47NYdAa\n5Vh7er3CJRuiFIT3uft/hNMj7r7VzP4uhm0/BpxgZseZ2QiCIbW/H8N6pQqp9U6tIsCMO/SOFD67\nc//JV7KQZXyZLowCX6aLhSzj/pNz9iUZ5VgXl8UZUkv+DHZNiXCU07JlT0W5HhVh3e8DngN+Dtww\n2PuVISRj2MPiKgLMuEPvqsLnRYt80zlBZmAWZAqbzhliaJuWasLiOENqyRRqzRDM7ApgPvDW8Au7\naDTwiLt/Mqki1R91TKsjpX+RFlXoZZv6c4jd+15DL5/Pg4jHuve9ed9fOUTNPZXNbAxwNEFuUNpp\nbJe7/08sraySCkKdiRBgNtoQ1IlRWNzQag6V3X2nu7/g7hd5MNz16wR3G40ys6z01ZS8ihhgVhUW\nl3+jV/iGrzp8jjH4TkRaYXHWj4sMSZShKz5oZpsJHozzH8ALBE9RExmaKgLMyGHxzJkwbdrBIlAo\nBPMzZ/Z5W1UhetZ77qYVFmf9uMiQRbnL6G+A04Dn3P04YBbwSKKtkvpmBkcd1fc69tKlwfxRR/W5\nlDF3LqxYEVzjNwt+rlgRLO9VKMDOnbBx48GiMG1aML9zZ58zhUjrg+BLbseOvl+cxS/WHTvS/4s4\navuqONaxblfyabDUmTCdBp4EmsLf/zNKYh33pLuM6kycvWx7etw7OvrePtTRESyvpX1Z7rlbTfvU\no7mhEWNP5TXAh4G/BcYCvwZOdfczkitTlSlUlgEVCtDcfHC+p6dvkDoUWQ9j02pf1o+L9BFnT+UP\nAXuAhcD9BLegfrC25onErFCAzs6+yzo7a7t1KM0wNq2wOIq0tivJi3IaAbQDZ4e/twGjo3wu7kmX\njKSinh73N74xuHQxdWowP3VqMP/GNw7tslESo39G7fgV5X1pjU6qUVFziYiXjFoGKxhm9icEg8sd\nAxwPvAW4jSBcFkmfGbSE/5RnzAjmZ8yAJ58Mlg/lUkZ/YSzUHsZCsK7Su3887AAW9X1xty+qtLYr\nw2OwigFsBEYAT5Qs+68o1SbuSWcI0q9CwX3Bgr5B54IFtf/FmkYYm1ZYXI20titDQoyh8k/c/V1m\n9oS7n2JmLQTjG01JuFYdQqGyDMhzEHRGbWMe9kVyI85Q+T/M7C+Aw83svcB3gR/U2kCpQXkRrzXM\ni7q+uLcbVYQeyFUFnUnsR5R1Rm1jNfsiEqMoBeE6YBvwX8DlwH3AXybZqKTE/czdVMTdSzTq+tLq\nnRqlB3KxLVF64yaxH3H2GK5mX0Ri1m9BKI5X5O4Fd/+qu1/g7h8Lf8/dv8rubpg3Lxjp0j34OW9e\nzopCaeAYRy/RqOuLe7tRRe2BHLU3bhL7EXWdUdsYd89ikWr0Fy5Q8hwE4K4ogUTSUy2hcr08Ozj2\nXqJJBJ1xqqYHcpSgM4n9SCIEVmgrMSKG5yE84e6nlP+eplpC5boa9jjuwDHrQWfcPZCT2I+0jo1I\nBHGEyt7P77mU6rOD4xR3eJr1oLN4mahUaaZQrWr3I85jWG07B9tuNe8TiaK/UwegB3gN2AUcCH8v\nzr8W5fQj7qmWS0ZVPToxq+J+HGLU9aXVO7X0clHxMlH5fDWq3Y+0egzH2aNZxKNfMuq3p7K7N/f3\nWh4Vhze+4QZ48cXgzGDJkgrDHmdZ1F6i7vH2dk2rd2pTE4wZAx0dsGFDML9hQ3CGMGZM9ZeNqtmP\nuI9hVFG3G/V9ItWIUjWyMqmnciju8DTrQWf5mUAtQ1q7V7e/WR5eWsNQS0TE1VM5S9RTuUquoLNm\naR3DqNvVf2OJIM6eypJHnlIInAflx6C/Y1LNMYy6zqjty3LQL/UrymlEViZdMopIQxT3L2oQG3eA\nH1XWg37JJeIa/lpySEMUV+ZVBLFxB/hRZT3ol7qmDKGelX8ZVfvlVI+Kl1mKX+DQ90u10vsHO4bV\nrjNqO6P8t9N/Y4kgaoaggiCNJ4kgVuGuZJhCZYkuaiAaZRjqJLYbpySCWIW7UidUEBpd1OGgowxD\nncR241R6aSeuoaWTWKdISlQQGllpIDrQ0M1Rh6GOe7txS2JoaQ1XLXVEGUKjixqIlhaBotIhJZLa\nbhKSCGIV7kqGKVSW6KIGonkYhlpEDqFQWaJxh4UL+y5buPDQyzaFAnR29l3W2dn/840Hmi8uUxAr\nkikqCI3MHU4/HZYvhwULgi/3BQuC+dNP75shvPnN8OSTMHVqcGYwdWow/+Y39y0KcT5fWESGVSoF\nwcwuMLNnzKxgZoOexkjKzKAl7NQ+Y0YwP2NGMN/SUv0zixXEimRTlPEt4p6Ak4G3AQ8C06N+TmMZ\nJaBQcF+wwPsMobxgQeWhlqO+L40ho0WkX+Rh+GszexC42t0jJcUKlRMSNdyN+30iMizqJlQ2s3lm\ntt7M1m/bti3t5uRLlJ7FUcPdat4XJaSuRtQezWn0fBapI4kVBDNbY2ZPV5g+VM163H2Fu0939+nj\nxo1Lqrn1J0rP4qjhbjXvixJSVyNqj+Y0ej6L1JnEhr9297OTWrcMorxncfE5xBs3Bp3JCoXgkk7W\nh1ouDalBzxcWSVqUoCGpCYXKyenpce/o6BvudnRUfh5xnM9Ujho+R6XnC4vUjCyHymZ2PvAVYByw\nA9jo7ucM9jmFylWKu2dxVHGHygqzRWqS6VDZ3e9x9/Hufpi7vzFKMci1NMLOYmZQqjRTGKg9tQbA\ncYbKcYfeItKvzN9llHtphJ2lA9F1dARnBh0dfUcrTaJ9cYfKcYfeIjIgPVM5SWmFnU1NMGZM39FI\ni8HymDEHL6tkPYzNeugtUm+iBA1ZmXIZKqcZdpYHyP0FynG2L+5QubjOgearfZ9IgyHLofJQ5TZU\nznrYmVYILCLDItOhckNJM+yMEhbH3T6FuyK5pYKQpDTDzjSGoVa4K5JrCpWTlPUevnG3T+GuSK4p\nQxgO5XfrDMfdO6V/rRf198ziuNuXxv6KSL/0TGVRuCsigEJlibvHsIjUPRWEehR3j2ERaQgqCCIi\nAqgg1CczWLfu4FlBU9PBs4V165QjiEhFCpXrmUJlEUGhslTTYziN4blFJHNUEOpRNT2G9SxiEQmp\np3I9itpjOOvDX4vIsFKGUM+i9BiupkeziOSSeipLdAqfReqaQuVaNFLIqvBZREIqCOUaKWRV+Cwi\nJVQQSpWGrMUvv+IX5o4d9fcb++gJAAAGv0lEQVQXcX/hc1dX/+FzIxwXkQalDKFcI4asCp9F6ppC\n5VooZK1Mx0UklxQqD5WeCVyZjotI3VNBKKVnAlem4yLSENRTuZSeCVyZjotIQ1CGUImeCVyZjotI\nLilDqEX5l5y+9AI6LiJ1rbEKQtSetuqRKyINqHEKQtSetuqRKyINqjEKQtSetuqRKyINLJVQ2cxu\nAj4I/Bb4OfApd98x2OdqCpWj9rRVj1wRqTOZ7qlsZrOBH7n7ATP7OwB3v3awz9V8l1HUnrbqkSsi\ndSTTdxm5+2p3PxDOPgqMH4aNRutpqx65ItKgspAh/DGwqr8XzWyema03s/Xbtm0b2hai9rRVj1wR\naWCJ9VQ2szXAmyq8dIO7/0v4nhuAA0B3f+tx9xXACgguGQ2xMdF62qpHrog0sNR6KpvZHwF/Csxy\n9z1RPhNLhhClp6165IpIHYmaIaQylpGZzQGuBWZELQYxbXjg+WrfJyJSR9LKEG4GRgP/amYbzey2\nlNohIiKhVM4Q3P330tiuiIj0Lwt3GYmISAaoIIiICKCCICIiIRWEWmiYbBGpIyoIQ6VhskWkzqgg\nDIWGyRaROpTKbae5VzqkxbJlB4fK1jDZIpJjqQ1dMRQ1D10RNw2TLSI5kOnhr+uChskWkTqjgjAU\nGiZbROqQMoSh0DDZIlKHlCHUQsNki0gOKEMYDhomW0TqiAqCiIgAKggiIhJSQRAREUAFQUREQioI\nIiICqCCIiEgoV/0QzGwbsCWBVb8BeCWB9eaRjkVAxyGg4xDI+3Fod/dxg70pVwUhKWa2PkqnjUag\nYxHQcQjoOAQa5TjokpGIiAAqCCIiElJBCKxIuwEZomMR0HEI6DgEGuI4KEMQERFAZwgiIhJSQRAR\nEUAFoZeZ3WRmz5rZU2Z2j5kdlXab0mBmF5jZM2ZWMLO6v82unJnNMbOfmdnzZnZd2u1Ji5l9w8x+\nbWZPp92WNJnZsWb272a2Kfz/oivtNiVJBeGgfwUmu/sU4Dng+pTbk5angY8AD6XdkOFmZs3APwDn\nApOAi8xsUrqtSs23gDlpNyIDDgBXufvJwGnAZ+r534QKQsjdV7v7gXD2UWB8mu1Ji7tvcvefpd2O\nlLwTeN7df+HuvwX+GfhQym1Khbs/BPxP2u1Im7v/yt0fD3/fBWwC3pJuq5KjglDZHwOr0m6EDLu3\nAC+VzG+ljv/nl+qY2UTgFOAn6bYkOS1pN2A4mdka4E0VXrrB3f8lfM8NBKeJ3cPZtuEU5Tg0qErP\nQNV92YKZjQLuAha6+2tptycpDVUQ3P3sgV43sz8CPgDM8jruoDHYcWhgW4FjS+bHAy+n1BbJCDNr\nJSgG3e5+d9rtSZIuGYXMbA5wLXCeu+9Juz2SiseAE8zsODMbAXwC+H7KbZIUmZkBXwc2ufuX0m5P\n0lQQDroZGA38q5ltNLPb0m5QGszsfDPbCpwO/NDMHki7TcMlvKngz4AHCMLD77j7M+m2Kh1m9m1g\nHfA2M9tqZp9Ou00p+X3gYuA94ffCRjN7X9qNSoqGrhAREUBnCCIiElJBEBERQAVBRERCKggiIgKo\nIIiISEgFQRqGmfWU3Dq4MRyKoNp1HGVm8+NvXe/6TzKzdWa2z8yuTmo7IpXotlNpGGb2G3cfVeM6\nJgL3uvvkKj/X7O49Ed73O0A78GHgVXf/4lDaKTIUOkOQhmZmzeGzMB4Ln4Vxebh8lJn9m5k9bmb/\nZWbFUU//Fjg+PMO4ycxmmtm9Jeu72cwuDX9/wcw+b2YPAxeY2fFmdr+ZbTCztWZ2Unl73P3X7v4Y\nsD/xnRcp01BjGUnDO9zMNoa//9Ldzwc+Dex091PN7DDgETNbTTDq6fnu/pqZvQF41My+D1xH8NyM\nDgAzmznINve6+7vD9/4b8KfuvtnM3gXcArwn7p0UGSoVBGkkrxe/yEvMBqaY2cfC+THACQQD3d1o\nZmcBBYJhsN84hG3eAb2jZZ4BfDcYHgeAw4awPpHEqCBIozPgs+7eZ8ym8LLPOGCau+83sxeAkRU+\nf4C+l17L37M7/NkE7KhQkEQyQxmCNLoHgCvCIY4xsxPN7AiCM4Vfh8XgDwiCXoBdBIMgFm0BJpnZ\nYWY2BphVaSPhGPq/NLMLwu2YmU1NZpdEhkZnCNLovgZMBB4PhzreRnCHTzfwAzNbD2wEngVw9+1m\n9kj48PlV7n6NmX0HeArYDDwxwLbmArea2V8CrQSP6Hyy9A1m9iZgPXAkUDCzhcCken4oi2SHbjsV\nERFAl4xERCSkgiAiIoAKgoiIhFQQREQEUEEQEZGQCoKIiAAqCCIiEvr/hb7RHhnNXGMAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb5bd668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值\n",
    "u = np.mean(X, axis=0)\n",
    "# 方差\n",
    "v = np.std(X, axis=0)\n",
    "\n",
    "X = (X - u) / v\n",
    "\n",
    "# 作图\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Normalization data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 280,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X加上偏置项\n",
    "X = np.hstack((np.ones((X.shape[0],1)), X))\n",
    "# 权重初始化\n",
    "w = np.random.randn(3,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显示初始化直线位置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 281,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEKCAYAAAAMzhLIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYFNXVx/HvYR02UQFXApjERJEg\nsohbBKMCalwwouACuKFEBH3VKHGBmGjcIi5EFNcoxF3cWUUFDESGRUTRYFwRo4gLCoLM9Hn/uD0w\nPfQ0PTPdUz09v8/z1DNT3berThfDPXXvrbpl7o6IiEh56kQdgIiI5DYlChERSUmJQkREUlKiEBGR\nlJQoREQkJSUKERFJSYlCRERSUqIQEZGUlChERCSlelEHkAktW7b0du3aRR2GiEiNsmDBgi/dvdXW\nyuVFomjXrh2FhYVRhyEiUqOY2UfplFPXk4iIpKREISIiKSlRiIhISnkxRpHMxo0bWbFiBevXr486\nlBqpoKCA1q1bU79+/ahDEZGI5W2iWLFiBc2aNaNdu3aYWdTh1CjuzurVq1mxYgW77bZb1OGISMTy\ntutp/fr1tGjRQkmiEsyMFi1aqDUmIkAeJwpASaIKdOxEpEReJwoRkax5/3245BKIxaKOJOuUKLKo\nbt26dOrUiQ4dOtCvXz/WrVtX4W2cddZZvP322wBce+21Ce8dcMABGYlTRCroiSdgn33gnnvgvfei\njibrlCiyqFGjRixevJilS5fSoEED7rzzzgpv45577qF9+/bAloniX//6V0biFJE0rV8Pw4ZBv36w\n556waBH84hdRR5V1ShRxEydCu3ZQp074OXFiZrf/61//mvfiZx4333wzHTp0oEOHDtxyyy0ArF27\nlqOOOoq9996bDh068OijjwLQs2dPCgsLueyyy/jhhx/o1KkTp5xyCgBNmzYF4KSTTuLFF1/ctK/B\ngwfz5JNPUlxczCWXXEK3bt3o2LEjd911V2a/lEht8t57cMAB8Pe/w0UXwaxZobKoBfL28tiKmDgR\nhgyBkp6hjz4K6wDxOrlKioqKmDx5Mn369GHBggXcf//9/Pvf/8bd6d69Oz169OD9999nl1124YUX\nXgDg22+/TdjGddddx9ixY1m8ePEW2+/fvz+PPvooRx55JD/++CMvvfQS48aN495776V58+bMnz+f\nDRs2cOCBB9KrVy9d8ipSUY8+CmefDfXrw7PPwtFHRx1RtVKLArj88s1JosS6deH1qihpAXTt2pU2\nbdpw5plnMmfOHPr27UuTJk1o2rQpxx9/PLNnz+ZXv/oVM2bM4NJLL2X27Nk0b9487f0cccQRzJw5\nkw0bNjB58mQOPvhgGjVqxLRp03jwwQfp1KkT3bt3Z/Xq1SxfvrxqX0qkNlm/HoYOhf79oUOH0NVU\ny5IEqEUBwMcfV+z1dJWMUZTm7knL/uIXv2DBggW8+OKLjBw5kl69enHVVVeltZ+CggJ69uzJ1KlT\nefTRRxkwYMCmfd1+++307t27al9EpDb6z3/gxBPhjTfgD3+Av/wltChqIbUogDZtKvZ6VRx88ME8\n/fTTrFu3jrVr1zJp0iR+/etfs3LlSho3bsypp57KxRdfzMKFC7f4bP369dm4cWPS7fbv35/777+f\n2bNnb0oMvXv3Zty4cZs+85///Ie1a9dm/kuJ5JuHH4YuXWDFCnjhBbj++lqbJEAtCgCuuSZxjAKg\ncePweqZ17tyZwYMHs++++wLh8td99tmHqVOncskll1CnTh3q16/PuHHjtvjskCFD6NixI507d2Zi\nmdH2Xr16MXDgQI455hgaNGiwadsffvghnTt3xt1p1aoVTz/9dOa/lEi++OEHGDEC7r4bDjwQHnkE\nWreOOqrIWXldITVJ165dveyDi5YtW8aee+6Z9jYmTgxjEh9/HFoS11yTmYHsmqyix1CkRnvnndDV\n9OabMHIkXH011Mvvc2kzW+DuXbdWLr+PQgWccooSg0itNWECnHsuNGoEkydDnz5RR5RTIh2jMLP7\nzOwLM1ta6rXRZvapmS2OL0dGGaOI5LF16+Css+C006BzZ1i8WEkiiagHsx8Akv2rjHH3TvHlxSTv\ni4hUzbJl0L073Hdf6HeeORN23TXqqHJSpF1P7j7LzNpFGYOI1EIPPhjuj2jSBKZOhcMPjzqinBZ1\ni6I8w8xsSbxraruogxGRPLF2LZx+OgwaBPvuG7qalCS2KhcTxTjgZ0An4DPgb8kKmdkQMys0s8JV\nq1ZVZ3wiUhO99VZIDv/4B1x1FcyYAbvsEnVUNULOJQp3/9zdi909BtwN7FtOufHu3tXdu7Zq1ap6\ng0yTmXHRRRdtWr/pppsYPXp0xvej6cdFUnCH+++Hbt1g9WqYPh3+9CeoWzfqyGqMnEsUZrZzqdW+\nwNLyymZU2ftJMnB/ScOGDXnqqaf48ssvq7ytVDT9uEg5vv8+dDOdcQbsv3/oajr00KijqnGivjz2\nYWAu8EszW2FmZwI3mNmbZrYEOAS4MOuBjB4NF164OTm4h/Uqnv3Xq1ePIUOGMGbMmC3eW7VqFb/7\n3e/o1q0b3bp147XXXtv0+uGHH07nzp0555xzaNu27aZEc9xxx9GlSxf22msvxo8fD6Dpx0XK8+ab\noRUxYUJoQUybBjvtFHVUNZO71/ilS5cuXtbbb7+9xWtJxWLuI0a4Q/iZbL2SmjRp4t9++623bdvW\nv/nmG7/xxht91KhR7u4+YMAAnz17tru7f/TRR77HHnu4u/t5553n1157rbu7T5482QFftWqVu7uv\nXr3a3d3XrVvne+21l3/55Zeb9lN2v+7uTz31lA8cONDd3Tds2OCtW7f2devW+V133eV//vOf3d19\n/fr13qVLF3///fe3iD/tYyiSS2Ix97vvdi8ocN9pJ/eZM6OOKGcBhZ5GHas7s82g5Iz/1lvDAmG+\nlzFjwvtVsM022zBw4EBuu+02GjVqtOn1GTNmbHrEKcCaNWv47rvvmDNnDpMmTQKgT58+bLfd5ou+\nbrvttk3vffLJJyxfvpwWLVqUu+8jjjiC4cOHs2HDBqZMmZIw/fiSJUt44okngPDsi+XLl+s5FVLz\nffdduMP6n/+Eww4LrYkdd4w6qhpPiQI2J4uSJAEZSRIlLrjgAjp37szpp5++6bVYLMbcuXMTkgeU\nPw35K6+8wowZM5g7dy6NGzemZ8+erF+/PuV+Nf241CpvvBHmanrvPfjzn8N8TRqwzoicG8yORMmY\nRGmlxyyqaPvtt+fEE0/k3nvv3fRar169GDt27Kb1kudWHHTQQTz22GMATJs2ja+//hoIZ/3bbbcd\njRs35p133mHevHmbPqvpx6VWc4fx48Nd1t9/H+6wvuIKJYkMUqIoSRK33hq6m2Kx8PPWWzOaLC66\n6KKEq59uu+02CgsL6dixI+3bt+fOO+8EYNSoUUybNo3OnTszefJkdt55Z5o1a0afPn0oKiqiY8eO\nXHnlley3336btlUy/fgpSWY17NWrF7NmzeKwww5LmH68ffv2dO7cmQ4dOnDOOedQVFSUke8pUq3W\nrIGTT4ZzzoGePcNVTT16RB1V3tE04xCubvrmm83dTSXJY9ttq3zlU0Vt2LCBunXrUq9ePebOncvQ\noUOTPie7OmiacclpixaFrqYPPghdTZdeCnV07lsRmma8IkaPDsmhZEyiZMwiQ2MUFfHxxx9z4okn\nEovFaNCgAXfffXe1xyCS09zhzjvDyVzLlvDKK3DQQVFHldeUKEqUTQoRJAmA3XffnUWLFkWyb5Gc\n9+23cPbZ8PjjcMQRYXK/li2jjirv5XU7LR+61aKiYyc5Z8GC8MyIp54Kz7B+/nkliWqSt4mioKCA\n1atXq8KrBHdn9erVFBQURB2KSOhqGjsWDjgAfvwRXn0V/vAHjUdUo7ztemrdujUrVqxAM8tWTkFB\nAa31UHmJ2jffwJlnhlbEUUeFmV9T3GQq2ZG3iaJ+/fq601ikJps/H046CT75BG68Ef7v/9SKiIiO\nuojkFvdwH9OBB0JxMcyeDRdfrCQRobxtUYhIDfT112FK8KefhmOOCc+R2H77qKOq9ZSiRSQ3vP46\n7LMPvPAC3HxzSBZKEjlBiUJEouUeEsOBB4b7l+bMCTfTRXQvk2xJXU8iEp2vvoLBg+G55+C44+C+\n+6DU1PqSG9SiEJFozJ0bupqmTAmD1089pSSRo5QoRKR6xWLhcteDDw5Tgb/2Ggwfrq6mHBb1M7Pv\nM7MvzGxpqde2N7PpZrY8/lOnGCL5YvXqcDXTH/4Axx4LCxeG51pLTou6RfEA0KfMa5cBL7n77sBL\n8XURqeleew06dYLp08OUHI8/Hqbyl5wXaaJw91nAV2VePhb4R/z3fwDHVWtQIpJZsViYxK9HD2jY\nMIxNnHeeuppqkKhbFMns6O6fAcR/7pCskJkNMbNCMyvUfE4iOWrVKvjtb+Gyy+B3vwtdTZ07Rx2V\nVFAuJoq0uPt4d+/q7l1btWoVdTgiUtbs2eGqppkzYdw4eOQR2GabqKOSSsjFRPG5me0MEP/5RcTx\niEhFxGJw7bVwyCHQuDHMmwfnnquuphosFxPFs8Cg+O+DgGcijEVEKuKLL8KT5y6/PDzPesGCMIAt\nNVrUl8c+DMwFfmlmK8zsTOA64HAzWw4cHl8XkVz36qshKbz6KowfDxMnQrNmUUclGRDpFB7uPqCc\ntw6t1kBEpPKKi0NX0+jR8POfhzutO3aMOirJIM31JCKV9/nncOqpMGMGnHJKGLRWKyLvKFGISOXM\nnBmSwzffwD33hOdIaMA6L+XiYLaI5LLiYvjTn+Cww8Kd1a+/Hp5rrSSRt9SiEJH0/e9/oRUxcyYM\nHAh//zs0bRp1VJJlShQikp6ScYjvvguPKB08OOqIpJqo60lEUisuhquugl69oGVLmD9fSaKWUYtC\nRMq3ciWcfHK4N+L00+H226FJk6ijkmqmRCEiyU2bFi59XbsW/vGPMCYhtZK6nkQkUVFRmIKjTx/Y\ncUcoLFSSqOXUohCRzT79FAYMCDO/nnVWeJZ148ZRRyURU6IQkWDKFDjtNPjhB5gwIVzhJIK6nkSk\nqAhGjgyzvu68c5jxVUlCSlGLQqQ2++ST0NX02mswZAjccgs0ahR1VJJjlChEaqsXXgiD1D/+CA8/\nDP37Rx2R5Ch1PYnUNhs3wh/+EJ5l/ZOfhOdYK0lICmpRiNQmH38cksLcuTB0KNx8MxQURB2V5Dgl\nCpHa4rnnYNCgMHj96KPhUaUiacjZricz+9DM3jSzxWZWGHU8IjXWjz/CRRfBMcfAbruFriYlCamA\nXG9RHOLuX0YdhEiN9eGHcNJJ4ZkRw4bBTTdBw4ZRRyU1TK4nChGprKefDhP5xWLw+ONwwglRRyQ1\nVM52PQEOTDOzBWY2JOpgRGqMH3+ECy6Avn3h5z+HRYuUJKRKcrlFcaC7rzSzHYDpZvaOu88qeTOe\nPIYAtGnTJqoYRXLLBx+Erqb582H4cLjhBnU1SZXlbIvC3VfGf34BTAL2LfP+eHfv6u5dW7VqFUWI\nIrnlqadgn31g+fLw+623KklIRuRkojCzJmbWrOR3oBewNNqoRHLUhg1w/vnwu9/BL38Zupr69o06\nKskjudr1tCMwycwgxPhPd58SbUgiOei//w1dTQsWwIUXwnXXQYMGUUcleSYnE4W7vw/sHXUcIjnt\n8cfDMyPq1oVnngn3SYhkQU52PYlICuvXw3nnhZvm2rcPXU1KEpJFShQiNcny5bD//nDHHXDxxTBr\nFrRtG3VUkudysutJRJJ45JHwzIj69cO8Tb/9bdQRSS2hFoVIrvvhBzj33PCAoV/9ChYvVpKQaqVE\nIZLL3n0X9tsP7rorPEPilVfCMyREqlG5icLMfmVm88zsEzMbb2bblXrv9eoJT6QW++c/oWtX+PTT\n8DS6668P3U5S40ycCO3aQZ064efEiVFHVDGpWhTjgNHAr4D/AHPM7Gfx9/TXKpItP/wAZ58Np5wC\nnTqFrqYjj4w6KqmkiRPD0NJHH4F7+DlkSM1KFqkSRVN3n+Lu37j7TcAwYIqZ7UeYsE9EMu2dd6B7\nd7jnHhg5El5+GVq3jjoqKUc6LYXLL4d16xJfW7cuvF5TpEoUZmbNS1bc/WXgd8BDgK7HE8m0hx4K\nXU2ffQZTpsC110I9XZiYq9JtKXz0UfLPl/d6uvuuzq6sVIniemDP0i+4+xLgUOCpbAYlUqusWwdn\nnAEDB0KXLqGrqXfvqKOqUaIYA4iqpRBJV5a71/ilS5cuLlIjvfWW+157uZu5X3GF+8aNUUeUUyZM\ncG/bNhyetm3DerIyjRu7h2ozLI0bl192a9tLl1niPksWs8RyycqULJXRtm3ybbVtW/FtAYWeRh0b\neSWfiUWJQqpbRiqc++8PNdoOO7hPm5bZAPPAhAnu9esnVob16295rNOtOCuSUNKR7n7TTSjpyuT2\nlChEsqTKFc7337sPGhQ+2LOn+8qVGT3TLYkxne1ler+ZjK9Fi+QVYosWieXSrTgzeSZe8h3S+Tto\n2jT5fps2rdx+1aJQopAaoEr/Ud98033PPUMtNmqUe1FRxs90J0xwr1MncXt16my5vai6bNLdb7pd\nNlGd2Zd8l60dl0zvN5N/LxlLFMAvgJeApfH1jsAV6Wy8uhYlCtmaKPqm3d0PPbTk/Zifzr3+Q51G\n7jvu6P7SS5vKZPpMt0mT5Ntr0iSxXK532aSbKNKNL9PHOV3Z2G+m/p4zmSheJTyGdFGp15ams/Hq\nWpQoJJVMnzmnWxGXJIkmfOcPcqo7+HQO9RMO+iyhXLoVYrrS3V6mz9jTlY1B4EwPemdSVPtNRyYT\nxfz4z9KJYnE6G6+uRYmiFovFUq975s+cy3brlCx16iSWA/cOLPFl/NKLqONXcLXXoWiLiq5u3eTb\nq1u3UkdkUwsmcXuxSu8304ks3X+PdMcoKiKXx2SikMlEMRn4GbAwvn4CMDmdjVdlAfoA7wLvAZel\nKqtEEb10/yNs7ooJy6GHVmF7o0b5st4jvG2bWCjXJubLeo8Iff+lZHqwM62KOBbzsxjv6yjwlezk\nPXi5ymf26RrFKB/DiFIxxnwMI3wUiccl3f2GhLLl961sIks3IU+Y4N6gQWK5Bg1yp5LNB5lMFD8F\nZgDrgE+BOUDbdDZe2QWoC/w3vu8GwBtA+/LKK1FkTyab9GWTRHnJYsIE98aNEiumxo1iiduLxZMC\nbKoUxxDWl/UekdCyCAmguMx+iys92Bkq4uFlKuLhmyviNWvcBwxwB5/K4d6Kz1NWxOm2UNISi/kt\nJD8ut5DsuGyZAMoel3QTT8n+U67H5fpVWbVFRhIF4c7tE+O/NwGapbPRqi7A/sDUUusjgZHllVei\nyI5MDxKmewY7pnnyimlM81GJ+22zuRIsWcYQWhilvbtzD1/I3qWSRbEvZG9/d+ceFf8esZjPpXt8\nX8M3JQmH8PqiRe677+5ep46P5Bq3LRLUlt83rRZKqf2nXHf3UVzlC+mUsMOFdPJRXJVQ7o2+o3xs\nvcTjPLbeCH+jb6njHIv5LVZO4rHExOOjRrmPKPVaLBbWy7TwJHdkskUxK50NZXKJd2/dU2r9NGBs\neeWVKEpJ84wu/b79rZ9xhjPxLctVanCyAmfE4bNFCRui7BhAcbF/XmfHeGW596Yk4RBeLy7eVHTo\n0OTfY+jQxPhKEkPZZQaHuDds6L7LLu6vvpre9rwCZ+yjRrkPH55YEQ8fnlgRx2J+b7MRSeO7t1mS\nSjz+ulHqcyPKHufkCRliCfst2d6mz5ddl5yTyURxJXAx8BNg+5IlnY1XdgH6JUkUt5cpMwQoBArb\ntGlT5QOWF03cdM/o0iw3upwKbHSZCizdFkC6LYp0WwqvcLB/TsuEcp/T0l/h4M2FYjG/pZyK/RaG\nJ1Rg4Xts2aVU9nskO2P/im3D7717u3/xxabj/FKH4V63Tthe3Toxf6nDlhX7uILkiXFcQZl/o+6h\nJbMpWQyPf6/u3RO+xxvHXulflDkuX9DS3zj2ysQDXboyL1mSVOolJwyJCXnLE4Z0tye5I5OJ4oMk\ny/vpbLyyS3V3PeXy5WtpS/eMrgLlSs4wy1ZgZc9M0x0rCGMUW55hJx2jKChO+AdpXFCc+O9RVLQp\nSXxOS4fEdS8q2lQ0JJ7EZDGG4YmJZ2tdSqW+79h6yRPPv+ocsHm/FajYJzwU81vrJFawt9YZ4RMe\nKnPGPjz5fhNaGcXF7p06JS/XqVNCC2rTdkuXSVKpT3godEmVLje2Xpn4KrA9yR01+s5soB7wPrBb\nqcHsvcorX9VEEdWNOO6efldRuttK54wuWaUzfPgW5f63x8FJz0z/t8fBidtr08Y31i9IKLexfoF7\n2ZbeqFH+xK7nJ5yxP7Hr+Vu2eK66yle3SazsVrfp5H5VYh/7mGZXJm1RjGmWeOY84cFiXxzvbipZ\nFrO3T3iwVKUZi/m9zZJXxPc2S+zuWdbr/KTllvU6f8tuoa1V7CUxPpRYwZZbCafx7+ZXXuneMvG4\neMuW4fWy29va30sFuqjUoqh5MtmiGJhsSWfjVVmAIwlP1vsvcHmqslVNFNm4tT8t2Rj8S+eMrkcP\n970TK07fe+/weoniYveddkp+YHbaafOZaVHRlpVS6cqp9Bn2rruG18+PV6jnxyvcXXdNfkZccgZc\ndj3ujb6jfKwNS9jnWBuWOBhbXBzuhE4W346JYxQTHor52LqJFfHYusO3PLPv0iX59sq0FNKu2DOZ\n4JNtK9k2KzKmkM7fqcYoaqRMJorbSy13x8/0n0hn49W11MgWRTb+Y6VT4aRbccZi7sOGJS83bFji\nNjdudC9IbFF4QUHilNkV6RK56qoty3Yq06IonWjKLuefn5h4Sm6l3nvvsF6SJJs0SdxvLObLeiVW\nxMt6lamICwvdt9su+Xco0+KpcMW+tS7DdFso6Rw/94qdqKTT8tVVTzVO1rqegObAsxX9XDaXah2j\niKKrqKLbSlXhVCRR7Ltv8nL77pv8zDTVGWzJvpNVYGX7zUvKli5XtkxFE0+yFlTZxJOqIi4udr/t\ntnDHV8l0oJmq2NM9Y09nzKOiJyCZ/HvOxvYkq7KZKOoDyyr6uWwumbg8dsJDscSrnpL1EadzeWJF\nZXLwL90KJ90E0K1b8nLduiXGedBBW84HUbdueL3sdy3bCih99l+6XDoJtEeP5ImndBdaia0lnlQV\ncZcu7scfH34/+mj3Sy7JXMVeunyqdff0//50Zi9pymTX03PAs/Hl+XjX0/XpbLy6lionikye0VVE\nNgb/0qlwioq2nEinRYuEK4XSPmMv3e1U0t1Udr0kjnS6iipyRpzpxJOsIj7xRPdtt3WvV8/9b39L\nfG9rxzlbJxZb229FykmtlslE0aPUciDQOp0NV+dSpUSRjT7ikvJbW49i8C/dBBCLbR58LruUHnx2\nD1c3JRujKH3VU+nB8bKD1KUHx90zO3ha2a6YWMx9zJjwSLW2bd3nzq3c8VaFLTksk4lii9ZD3rUo\n0j3jTGdw0j3jN75lVCwW7hxOlgB22WXLQdGOHRPLdOy45aCoe2g5lC6X7NnPJV1FJUmhJFkk6yrK\n5OBpRY/zV1+5H3ts+B7HHhvWRfJQJhPFwiSvLUln49W1ZGQKj3TGCtJJFFEPJm5NNgZFK9KFlmxQ\nuqrfJ9V6RcvNmxdaEPXrhxaFWgCSx6qcKIChwJvAWmBJqeUDYEI6G6+upVpaFBXpesrG2EMmZXJQ\nNKoutEyLxcIYRL167u3aub/+etQRiWRdJhJFc6Ad8DDQttSS1XmeKrNU2xhFRa9i2VoLJUqZPBOv\n6VfZrF4drmYC97593b/+OuqIRKpFuomiHuVw92+Bb4EBAGa2A1AANDWzpu7+cXmfrVHMYNttYcQI\nGDMmrI8ZE97bdtuwXlKuTx/o3h1uuSWs33JLeG+77TaXg5AaLrwwcT8XXrh5+7mgbBzlxZVOudGj\nw3cufaxy6bumMncunHQSfP453HYbDBtWM+IWqU5byyTA0cByQhfUB0AMeCudLFRdS8bGKFKtp1su\nX7pi8l1xsfsNN4Supp/+NNxxLVLLUNUWRSl/AfYDZrj7PmZ2CPFWRl7J1Bl2ui0Uic6XX8KgQfDi\ni3DCCXDPPdC8edRRieSsdBLFRndfbWZ1zKyOu79sZtdnPbKarCZ3xeS7OXOgf39YtQr+/ncYOlT/\nLiJbkU6i+MbMmgKzgYlm9gVQlN2w8kC6LRSpHrEY3HADXHEFtGsXxiY6d446KpEaoU4aZY4F1gEX\nAFMI034fnc2gRDJq1So46igYOTJ0NS1cqCQhUgFbbVG4+1ozawvs7u7/MLPGQN3shyaSAbNmwYAB\nsHo1jBsH55yj1p1IBW21RWFmZwNPAHfFX9oVeDqbQYlUWSwG11wDhxwCTZrAvHlw7rlKEiKVkE7X\n03mEyQDXALj7cmCHbAYlUiVffBHuebniinCPxIIF0KlT1FGJ1FjpDGZvcPcfLX4mZmb1AM9qVCKV\n9corcPLJ8PXXMH48nHWWWhEiVZROi+JVM/sj0MjMDgceJzyjIivMbLSZfWpmi+PLkdnal+SR4mK4\n+mo49FDYZhv497/h7LOVJEQyIJ0WxWXAmYQJAs8BXgTuyWZQwBh3vynL+5B88fnncMop8NJLcOqp\nYdC6adOooxLJG+UmCjNr4+4fu3sMuDu+iOSWmTNDV9OaNXDvvXD66WpFiGRYqq6nTVc2mdmT1RBL\nacPMbImZ3Wdm2yUrYGZDzKzQzApXrVpVzeFJ5IqLYdQoOOww2H57eP11OOMMJQmRLEiVKEr/j/tp\nJndqZjPMbGmS5VhgHPAzoBPwGfC3ZNtw9/Hu3tXdu7Zq1SqT4Umu++wzOPzwMCYxcCDMnw8dOkQd\nlUjeSjVG4eX8XmXuflg65czsbuD5TO5barjp08M4xPffw/33w+DBUUckkvdStSj2NrM1ZvYd0DH+\n+xoz+87M1mQrIDPbudRqX2BptvYlNUhREVx5JfTuDS1bhlaEkoRItUj14KKopum4wcw6EVoxHxKu\ntJLabOXKMA3HrFlhHOL226Fx46ijEqk10rk8tlq5+2lRxyA5ZOpUOO00WLcOHnww/C4i1SqdG+5E\nql9REfzxj2Eqjh13hMJCJQlFOVx5AAAOIklEQVSRiORci0KEFStCV9OcOeHu6ltvhUaNoo5KpNZS\nopDcMnlyaDls2AATJ4ab6UQkUup6ktywcSNceikceSS0bh1mfFWSEMkJalFI9D75JDzH+l//Cg8W\nGjNGXU0iOUSJQqL1/PMwaFBoUTz8cEgYIpJT1PUk0di4ES65BI4+Gtq0CV1NShIiOUktCql+H30U\nksK8efD738Pf/gYFBVFHJSLlUKKQ6vXss2HqjeJieOwx6Ncv6ohEZCvU9STV48cf4f/+D449Fnbb\nDRYuVJIQqSHUopDs+/BDOOmk8MyI88+HG2+Ehg2jjkpE0qREIdn19NPhqXPu8OSTcPzxUUckIhWk\nrifJjg0bYMQI6NsXdt89dDUpSYjUSGpRSOa9/z6ceGK45PWCC+D666FBg6ijEpFKUqKQzHryyfDM\niDp1YNIkOO64qCMSkSpS15NkxoYNYaD6hBNgjz1g0SIlCZE8oUQhVffee3DAATB2bLgEdvZsaNcu\n6qhEJEMiSRRm1s/M3jKzmJl1LfPeSDN7z8zeNbPeUcQnFfDYY9C5M3zwQbiZ7m9/03iESJ6JqkWx\nFDgemFX6RTNrD/QH9gL6AHeYWVTP7pZU1q8P02+cdBLstRcsXhzmbRKRvBNJonD3Ze7+bpK3jgUe\ncfcN7v4B8B6wb/VGJ1u1fDnsvz+MGxcm9ps1K0zsJyJ5KdeuetoVmFdqfUX8NckVjzwSHk/aoEGY\nIvyoo6KOSESyLGuJwsxmADsleetyd3+mvI8lec3L2f4QYAhAG53NZt8PP4R7IsaPhwMPDM+O+MlP\noo5KRKpB1hKFux9WiY+tAErXPq2BleVsfzwwHqBr165Jk4lkyLvvhhvoliyByy6Dq6+G+vWjjkpE\nqkmuXR77LNDfzBqa2W7A7sDrEcdUu02cCF26wMqV8OKL8Ne/KkmI1DJRXR7b18xWAPsDL5jZVAB3\nfwt4DHgbmAKc5+7FUcRY661bB2edBaeeGi5/XbwYjjgi6qhEJAKRDGa7+yRgUjnvXQNcU70RSYJl\ny0JX01tvwR//CH/6E9TLteseRKS66H+/JHrwQRg6FJo0gSlToFevqCMSkYjl2hiFRGXt2jCZ36BB\n0K1b6GpSkhARlCgE4O23Yd994YEH4MorYcYM2GWXqKMSkRyhrqfa7oEHwlQczZrBtGlwWGWuahaR\nfKYWRW31/fehm+n002G//UJXk5KEiCShRFEbLV0axiEeeghGj4bp02HnnaOOSkRylLqeahN3uO8+\nGDYMtt02jEX85jdRRyUiOU4titri++/htNPCTXQHHRS6mpQkRCQNShS1wZIlYRqOhx8O8zRNmQI7\n7hh1VCJSQ6jrKZ+5w913w4gRsN12MHMm9OgRdVQiUsOoRZGv1qyBk0+Gc86Bgw8OXU1KEiJSCUoU\n+WjxYujaNTzP+pprYPJk2GGHqKMSkRpKiSKfuIfHk+63X5j99ZVXwqR+dfTPLCKVpxokX3z7LfTv\nH+6yPuQQWLQIfv3rqKMSkTygRJEPFi4MVzU9+SRcdx288AK0ahV1VCKSJ5QoajJ3GDsW9t8fNmyA\nV1+FSy9VV5OIZJRqlJrq22+hXz84/3w4/PAwgH3ggVFHJSJ5KKpHofYzs7fMLGZmXUu93s7MfjCz\nxfHlzijiy3mFhbDPPvDMM3DjjfDss9CiRdRRiUieiuqGu6XA8cBdSd77r7t3quZ4agZ3uP12uPhi\n2GknmDUrdDuJiGRRVM/MXgZgZlHsvmb6+ms480yYNAmOPjo8R2L77aOOSkRqgVwco9jNzBaZ2atm\npus7AV5/HTp3hueeg5tvDl1OShIiUk2y1qIwsxnATkneutzdnynnY58Bbdx9tZl1AZ42s73cfU2S\n7Q8BhgC0adMmU2HnFne45ZZwJdMuu8CcOdC9e9RRiUgtk7VE4e4Vflyau28ANsR/X2Bm/wV+ARQm\nKTseGA/QtWtXr1q0Oeirr8LT5559Fo47LjxHYrvtoo5KRGqhnOp6MrNWZlY3/vtPgd2B96ONKgLz\n5oWrmiZPhltvhaeeUpIQkchEdXlsXzNbAewPvGBmU+NvHQwsMbM3gCeAc939qyhijEQsBjfdFKbe\nqFsXXnsNhg8HDfqLSISiuuppEjApyetPAk9Wf0Q5YPVqGDwYnn8ejj8e7r03PK5URCRiOdX1VGv9\n61+hq2natHCfxBNPKEmISM5QoohSLAY33BAeLFS/fkgYw4apq0lEcooehRqVL7+EgQPDgHW/fuGR\npc2bRx2ViMgWlCiiMGdOeHbEl1/CHXfAueeqFSEiOUtdT9UpFoO//hV69oRGjWDuXBg6VElCRHKa\nWhTVZdUqOO00mDo1tCbuugu22SbqqEREtkqJojrMmgUDBoRLYO+6C84+W60IEakx1PWUTcXF8Je/\nhGdYN20K//43DBmiJCEiNYpaFNny+edw6qkwYwacfDLceSc0axZ1VCIiFaZEkQ0vvxySwzffhMte\nzzxTrQgRqbHU9ZRJxcVw9dVw2GHhzurXX4ezzlKSEJEaTS2KTPnf/+CUU2DmzHB10x13hHEJEZEa\nTokiE156KSSJNWvCcyMGD1YrQkTyhrqeqqK4GEaNgsMPD48mnT8/PGxISUJE8ohaFJX12WdhwPqV\nV0ILYuxYaNIk6qhERDJOiaIypk8Pl75+/z088AAMGhR1RCIiWaOup4ooKoIrroDevaFVq9DVpCQh\nInkuqkeh3mhm75jZEjObZGbblnpvpJm9Z2bvmlnvKOJL6tNP4Te/gWuugTPOCJe+tm8fdVQiIlkX\nVYtiOtDB3TsC/wFGAphZe6A/sBfQB7jDzOpGFONmU6ZAp06wcCE89BDccw80bhx1VCIi1SKSROHu\n09y9KL46D2gd//1Y4BF33+DuHwDvAftGESMQuppGjoQjjoCdd4bCwjA2ISJSi+TCYPYZwKPx33cl\nJI4SK+KvVb/vvoMjjwwPGTr7bLj11vAMCRGRWiZricLMZgA7JXnrcnd/Jl7mcqAImFjysSTlvZzt\nDwGGALRp06bK8W6haVP42c/g978PU4SLiNRSWUsU7n5YqvfNbBDwW+BQdy9JBiuAn5Qq1hpYWc72\nxwPjAbp27Zo0mVSJWbj0VUSklovqqqc+wKXAMe6+rtRbzwL9zayhme0G7A68HkWMIiISRDVGMRZo\nCEy3MN3FPHc/193fMrPHgLcJXVLnuXtxRDGKiAgRJQp3/3mK964BrqnGcEREJAXdmS0iIikpUYiI\nSEpKFCIikpIShYiIpKREISIiKdnme91qLjNbBXyUpc23BL7M0rZrEh2HQMch0HEIavpxaOvurbZW\nKC8SRTaZWaG7d406jqjpOAQ6DoGOQ1BbjoO6nkREJCUlChERSUmJYuvGRx1AjtBxCHQcAh2HoFYc\nB41RiIhISmpRiIhISkoUaTCzG83sHTNbYmaTzGzbqGOKgpn1M7O3zCxmZnl/pUdZZtbHzN41s/fM\n7LKo44mCmd1nZl+Y2dKoY4mSmf3EzF42s2Xx/xMjoo4pm5Qo0jMd6ODuHYH/ACMjjicqS4HjgVlR\nB1LdzKwu8HfgCKA9MMDM2kcbVSQeAPpEHUQOKAIucvc9gf2A8/L570GJIg3uPs3di+Kr8whP3qt1\n3H2Zu78bdRwR2Rd4z93fd/cfgUeAYyOOqdq5+yzgq6jjiJq7f+buC+O/fwcsA3aNNqrsUaKouDOA\nyVEHIdVuV+CTUusryOOKQdJnZu2AfYB/RxtJ9kT1hLucY2YzgJ2SvHW5uz8TL3M5ock5sTpjq07p\nHIdaypK8pksGazkzawo8CVzg7muijidblCji3P2wVO+b2SDgt8ChnsfXFG/tONRiK4CflFpvDayM\nKBbJAWZWn5AkJrr7U1HHk03qekqDmfUBLgWOcfd1UccjkZgP7G5mu5lZA6A/8GzEMUlEzMyAe4Fl\n7n5z1PFkmxJFesYCzYDpZrbYzO6MOqAomFlfM1sB7A+8YGZTo46pusQvZhgGTCUMXD7m7m9FG1X1\nM7OHgbnAL81shZmdGXVMETkQOA34TbxOWGxmR0YdVLbozmwREUlJLQoREUlJiUJERFJSohARkZSU\nKEREJCUlChERSUmJQgQws+JSlzkujk/LUNFtbGtmv898dJu2v4eZzTWzDWZ2cbb2I1KWLo8VAczs\ne3dvWsVttAOed/cOFfxcXXcvTqPcDkBb4Djga3e/qTJxilSUWhQi5TCzuvFnkcyPP4vknPjrTc3s\nJTNbaGZvmlnJLLLXAT+Lt0huNLOeZvZ8qe2NNbPB8d8/NLOrzGwO0M/MfmZmU8xsgZnNNrM9ysbj\n7l+4+3xgY9a/vEgpmutJJGhkZovjv3/g7n2BM4Fv3b2bmTUEXjOzaYRZZPu6+xozawnMM7NngcsI\nzy3pBGBmPbeyz/XuflC87EvAue6+3My6A3cAv8n0lxSpDCUKkeCHkgq+lF5ARzM7Ib7eHNidMEHg\ntWZ2MBAjTDe+YyX2+ShsmoH0AODxMIUQAA0rsT2RrFCiECmfAee7e8KcVvHuo1ZAF3ffaGYfAgVJ\nPl9EYvdu2TJr4z/rAN8kSVQiOUFjFCLlmwoMjU8njZn9wsyaEFoWX8STxCGEAWaA7wiTR5b4CGhv\nZg3NrDlwaLKdxJ9j8IGZ9Yvvx8xs7+x8JZGKU4tCpHz3AO2AhfFppVcRrjiaCDxnZoXAYuAdAHdf\nbWavmdlSYLK7X2JmjwFLgOXAohT7OgUYZ2ZXAPUJj1p9o3QBM9sJKAS2AWJmdgHQPp8fmCO5QZfH\niohISup6EhGRlJQoREQkJSUKERFJSYlCRERSUqIQEZGUlChERCQlJQoREUlJiUJERFL6fz1D/pA8\nJQ+TAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb5e40f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代更新训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第   0次更新，分类错误的点个数：  25\n",
      "第 1次更新，分类错误的点个数：25\n",
      "第 2次更新，分类错误的点个数：25\n",
      "第 3次更新，分类错误的点个数：25\n",
      "第   4次更新，分类错误的点个数：  23\n",
      "第   5次更新，分类错误的点个数：  22\n",
      "第   6次更新，分类错误的点个数：  18\n",
      "第   7次更新，分类错误的点个数：  14\n",
      "第 8次更新，分类错误的点个数：14\n",
      "第   9次更新，分类错误的点个数：  11\n",
      "第  10次更新，分类错误的点个数：   9\n",
      "第11次更新，分类错误的点个数： 9\n",
      "第12次更新，分类错误的点个数： 9\n",
      "第13次更新，分类错误的点个数： 9\n",
      "第14次更新，分类错误的点个数： 9\n",
      "第15次更新，分类错误的点个数： 9\n",
      "第  16次更新，分类错误的点个数：   8\n",
      "第17次更新，分类错误的点个数： 8\n",
      "第18次更新，分类错误的点个数： 8\n",
      "第19次更新，分类错误的点个数： 8\n",
      "第  20次更新，分类错误的点个数：   7\n",
      "第21次更新，分类错误的点个数： 7\n",
      "第22次更新，分类错误的点个数： 7\n",
      "第23次更新，分类错误的点个数： 7\n",
      "第24次更新，分类错误的点个数： 7\n",
      "第25次更新，分类错误的点个数： 7\n",
      "第26次更新，分类错误的点个数： 7\n",
      "第27次更新，分类错误的点个数： 7\n",
      "第28次更新，分类错误的点个数： 7\n",
      "第29次更新，分类错误的点个数： 7\n",
      "第30次更新，分类错误的点个数： 7\n",
      "第31次更新，分类错误的点个数： 7\n",
      "第32次更新，分类错误的点个数： 7\n",
      "第33次更新，分类错误的点个数： 7\n",
      "第34次更新，分类错误的点个数： 7\n",
      "第35次更新，分类错误的点个数： 7\n",
      "第36次更新，分类错误的点个数： 7\n",
      "第37次更新，分类错误的点个数： 7\n",
      "第38次更新，分类错误的点个数： 7\n",
      "第39次更新，分类错误的点个数： 7\n",
      "第40次更新，分类错误的点个数： 7\n",
      "第41次更新，分类错误的点个数： 7\n",
      "第42次更新，分类错误的点个数： 7\n",
      "第43次更新，分类错误的点个数： 7\n",
      "第44次更新，分类错误的点个数： 7\n",
      "第45次更新，分类错误的点个数： 7\n",
      "第46次更新，分类错误的点个数： 7\n",
      "第47次更新，分类错误的点个数： 7\n",
      "第48次更新，分类错误的点个数： 7\n",
      "第49次更新，分类错误的点个数： 7\n",
      "第50次更新，分类错误的点个数： 7\n",
      "第51次更新，分类错误的点个数： 7\n",
      "第52次更新，分类错误的点个数： 7\n",
      "第53次更新，分类错误的点个数： 7\n",
      "第54次更新，分类错误的点个数： 7\n",
      "第55次更新，分类错误的点个数： 7\n",
      "第56次更新，分类错误的点个数： 7\n",
      "第57次更新，分类错误的点个数： 7\n",
      "第58次更新，分类错误的点个数： 7\n",
      "第59次更新，分类错误的点个数： 7\n",
      "第60次更新，分类错误的点个数： 7\n",
      "第61次更新，分类错误的点个数： 7\n",
      "第62次更新，分类错误的点个数： 7\n",
      "第63次更新，分类错误的点个数： 7\n",
      "第64次更新，分类错误的点个数： 7\n",
      "第65次更新，分类错误的点个数： 7\n",
      "第66次更新，分类错误的点个数： 7\n",
      "第67次更新，分类错误的点个数： 7\n",
      "第68次更新，分类错误的点个数： 7\n",
      "第69次更新，分类错误的点个数： 7\n",
      "第70次更新，分类错误的点个数： 7\n",
      "第71次更新，分类错误的点个数： 7\n",
      "第72次更新，分类错误的点个数： 7\n",
      "第73次更新，分类错误的点个数： 7\n",
      "第74次更新，分类错误的点个数： 7\n",
      "第75次更新，分类错误的点个数： 7\n",
      "第76次更新，分类错误的点个数： 7\n",
      "第77次更新，分类错误的点个数： 7\n",
      "第78次更新，分类错误的点个数： 7\n",
      "第79次更新，分类错误的点个数： 7\n",
      "第80次更新，分类错误的点个数： 7\n",
      "第81次更新，分类错误的点个数： 7\n",
      "第82次更新，分类错误的点个数： 7\n",
      "第83次更新，分类错误的点个数： 7\n",
      "第84次更新，分类错误的点个数： 7\n",
      "第85次更新，分类错误的点个数： 7\n",
      "第86次更新，分类错误的点个数： 7\n",
      "第87次更新，分类错误的点个数： 7\n",
      "第88次更新，分类错误的点个数： 7\n",
      "第89次更新，分类错误的点个数： 7\n",
      "第90次更新，分类错误的点个数： 7\n",
      "第91次更新，分类错误的点个数： 7\n",
      "第92次更新，分类错误的点个数： 7\n",
      "第93次更新，分类错误的点个数： 7\n",
      "第94次更新，分类错误的点个数： 7\n",
      "第95次更新，分类错误的点个数： 7\n",
      "第96次更新，分类错误的点个数： 7\n",
      "第97次更新，分类错误的点个数： 7\n",
      "第98次更新，分类错误的点个数： 7\n",
      "第99次更新，分类错误的点个数： 7\n"
     ]
    }
   ],
   "source": [
    "for i in range(100):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    \n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        r = np.random.choice(num_fault)        # 随机选择一个错误分类点\n",
    "        t = np.where(y != y_pred)[0][r]\n",
    "        w2 = w + y[t] * X[t, :].reshape((3,1))\n",
    "        \n",
    "        s = np.dot(X, w2)\n",
    "        y_pred = np.ones_like(y)\n",
    "        loc_n = np.where(s < 0)[0]\n",
    "        y_pred[loc_n] = -1\n",
    "        num_fault2 = len(np.where(y != y_pred)[0])\n",
    "        if num_fault2 <num_fault:\n",
    "            w = w2        # 犯的错误点更少，则更新w，否则w不变\n",
    "            print('第%4d次更新，分类错误的点个数：%4d' % (i, num_fault2))\n",
    "        else:\n",
    "            print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xt4VOW9L/DvLxcIAUREvLSYQJV7\nRExArFsNVoqgPbZosVpaar3gAVuDBR/1aIXdbjzt0S1CPYARd3v6kO6j9XaqBaFofQSLPQZENxUE\nbyBbewypQeQmyfzOH+9M5pKZ5J3MWvOumfl+nmeeZCZr1npnKes3a33X+76iqiAiIipy3QAiIgoG\nFgQiIgLAgkBERGEsCEREBIAFgYiIwlgQiIgIAAsCERGFsSAQEREAFgQiIgorcd2AdJx44ok6ePBg\n180gIsopmzdv3qeqA7taLqcKwuDBg9HY2Oi6GUREOUVEdtssx0tGREQEgAWBiIjCWBCIiAhAjmUI\nyRw7dgx79+7FkSNHXDclJ5WVlWHQoEEoLS113RQiciznC8LevXvRt29fDB48GCLiujk5RVXR3NyM\nvXv3YsiQIa6bQ0SO5fwloyNHjmDAgAEsBt0gIhgwYADProgIQB4UBAAsBhngviPyTkMDMHgwUFRk\nfjY0uG5RenL+khERURA0NACzZgGHDpnnu3eb5wAwY4a7dqUjL84QXCsuLsbYsWNRVVWF6dOn41Dk\n/4g03HDDDXjrrbcAAPfee2/c38477zxP2klE/rnrrmgxiDh0yLyeK1gQPNCrVy9s3boV27ZtQ48e\nPbBixYq017Fy5UqMGjUKQMeC8Je//MWTdhKRf/bsSe/1ICq4guD3Nb4LLrgA77zzDgDggQceQFVV\nFaqqqvDggw8CAA4ePIjLLrsMZ511FqqqqvDYY48BACZOnIjGxkbccccdOHz4MMaOHYsZ4fPMPn36\nAAC+853vYPXq1e3buvbaa/Hkk0+ira0Nt912G8aPH48xY8bg4Ycf9vZDEVGXKirSez2QVDVnHjU1\nNZrorbfe6vBaKqtWqZaXqwLRR3m5eT0TvXv3VlXVY8eO6eWXX67Lli3TxsZGraqq0s8//1wPHDig\no0aN0i1btugTTzyhN9xwQ/t7W1paVFW1trZWX3vttbj1Ja7/qaee0pkzZ6qq6tGjR3XQoEF66NAh\nffjhh/XnP/+5qqoeOXJEa2pq9L333rNufzr7kIiS8+v44gUAjWpxjC2oMwS/rvFFvtGPGzcOFRUV\nuP7667Fx40ZMmzYNvXv3Rp8+fXDFFVdgw4YNOPPMM7F+/Xrcfvvt2LBhA/r162e9nalTp+LFF1/E\n0aNHsWbNGlx44YXo1asX1q1bh9/+9rcYO3YsJkyYgObmZuzatSuzD0VEaZkxA6ivByorARHzs74+\ndwJloMDuMvLrGl8kQ4hlinJHw4YNw+bNm7F69WrceeedmDx5Mu655x6r7ZSVlWHixIlYu3YtHnvs\nMVxzzTXt2/rVr36FSy65JLMPQkQZmTEjtwpAooI6Q8jmNb4LL7wQzzzzDA4dOoSDBw/i6aefxgUX\nXICPPvoI5eXl+N73vof58+djy5YtHd5bWlqKY8eOJV3v1VdfjV//+tfYsGFDewG45JJLsHz58vb3\n7Ny5EwcPHvT+QxFRXiuoM4RFi+LvEwaA8nLzuteqq6tx7bXX4pxzzgFgbis9++yzsXbtWtx2220o\nKipCaWkpli9f3uG9s2bNwpgxY1BdXY2GhNR78uTJmDlzJi6//HL06NGjfd0ffPABqquroaoYOHAg\nnnnmGe8/FBHlNUl1aSOIxo0bp4kT5Gzfvh0jR460XkdDg8kM9uwxZwaLFuX2KZ4X0t2HRJTcnDkm\nN2hrA4qLzRfQZctctwoQkc2qOq6r5QrqDAHI/Wt8RBRMc+YAsSf8bW3R50EoCjYKKkMgIvJLfX16\nrwcRCwIRkQfa2tJ7PYhYEIiIPFBcnN7rQcSCQETUBZshbyIjm9q+HkQFFyoTEaXDdljrSHAcxLuM\nbDk7QxCR00TkzyKyXUT+JiJ1rtqSKRHBvHnz2p/ff//9WLhwoefb4bDYRNmXzpA3y5YBra1mJKPW\n1twqBoDbS0atAOap6kgA5wK4WURG+b7VxH4XHvTD6NmzJ5566ins27cv43V1hsNiE2VfPgxrbctZ\nQVDVj1V1S/j3AwC2A/iyrxtduBC49dZoEVA1zzP8Nl9SUoJZs2Zh8eLFHf7W1NSEK6+8EuPHj8f4\n8ePxyiuvtL/+9a9/HdXV1bjppptQWVnZXlC+9a1voaamBqNHj0Z9+J41DotN5EZeDGtty2ZIVL8f\nAAYD2APguCR/mwWgEUBjRUVFh2FdrYduDoVU6+rMmLR1dcmfd1Pv3r11//79WllZqS0tLXrffffp\nggULVFX1mmuu0Q0bNqiq6u7du3XEiBGqqnrzzTfrvffeq6qqa9asUQDa1NSkqqrNzc2qqnro0CEd\nPXq07tu3r307idtVzXxYbA5/TZRakIe1tgXL4a+dh8oi0gfAkwDmqupniX9X1XoA9YAZuiKDDQGR\nb/BLlpgHANTVmdcznGz+uOOOw8yZM7F06VL06tWr/fX169e3T40JAJ999hkOHDiAjRs34umnnwYA\nTJkyBf37929fZunSpe1/+/DDD7Fr1y4MGDAg5banTp2KW265BUePHsXzzz8fNyz2m2++iSeeeAIA\nsH//fuzatQtDhgzJ6LMSFZJIcFwIQ944LQgiUgpTDBpU9aksbNAc/CPFAPCkGETMnTsX1dXV+OEP\nf9j+WigUwqZNm+KKBJB6eOyXXnoJ69evx6ZNm1BeXo6JEyfiyJEjnW6Xw2IT+atQhrxxeZeRAHgU\nwHZVfSArG41kBrFiM4UMnXDCCbjqqqvw6KOPtr82efJkPPTQQ+3PI/MmnH/++Xj88ccBAOvWrcOn\nn34KwHyL79+/P8rLy7Fjxw68+uqr7e/lsNhE5CeXdxn9E4DvA/iaiGwNPy71bWuRYrBkiblMFAqZ\nn0uWeFoU5s2bF3e30dKlS9HY2IgxY8Zg1KhRWLFiBQBgwYIFWLduHaqrq7FmzRqceuqp6Nu3L6ZM\nmYLW1laMGTMGP/3pT3Huuee2rysyLPaMJF9VJk+ejJdffhmTJk2KGxZ71KhRqK6uRlVVFW666Sa0\ntrZ68jmJKP8U1vDXCxcCLS3Ry0SRInH88RnfaZSuo0ePori4GCUlJdi0aRNmz57dYda1bOHw10TB\n5NVw/Rz+OpmFC00RiGQGkUzBowwhHXv27MFVV12FUCiEHj164JFHHsl6G4gouGx7SHupsAoC0PHg\n76AYAMDQoUPx+uuvO9k2EQVfZz2k/SoIeTG4XS5d9goa7juiYHLRQzrnC0JZWRmam5t5YOsGVUVz\nczPKyspcN4WIErjoIZ3zl4wGDRqEvXv3oqmpyXVTclJZWRkGDRrkuhlEVgppTvRFi+IzBAAoLzev\n+yXnC0JpaSl73hIVABchq0suekjn/G2nRFQYBg82RSBRZSXwwQfZbk1usb3tNOczBCIqDIU0DLUr\nLAhElBMKahhqR1gQiCgnLFpkQtVYfoeshYYFgYhywowZZr7iykrTn7Sy0jzPx0DZlZy/y4iICkeh\nDEPtCs8QiIgIAAsCERGFsSAQkXMNDaafQVGR+dnQEKz1BX27XmGGQEROed0D2VWP5nzoSc2eykTk\nlNc9kF31aA5yT2r2VCainOB1D2RXPZrzoSc1CwIROeV1D2RXPZrzoSc1CwJRJ3I9JMwFixYBpaXx\nr5WWdr8HsqsezfnQk5oFgSiFSEi4e7eZijsSErIoeM/LmW1d9WjOh57UDJWJUghySJhPuJ/9x1CZ\nKEP5EBLmAu7n4GBBIEohH0LCXMD9HBwsCEQp5ENIGBHknrv5tJ9znqrmzKOmpkaJsmnVKtXKSlUR\n83PVKtctSt+qVarl5aomGjeP8nL/P0s6282H/RxkABrV4hjLUJkoz7HnLjFUJiIA7LlL9lgQiPIc\ne+6SLRYEoixyMcxzLvTcZY/wgLAJGoLyYKhMuczrcDcXQlub7boKvQsJGCoTBUu+DPPstXz5HEGW\nE6GyiPybiHwiIttctoMoG/JlmGev5cvnyAeuM4TfAJjiuA1EWZEvwzx7LV8+Rz5wWhBU9WUA/3DZ\nBqJs8TrcXbTIhLCxiooyC4tdhLvsqRwcrs8QiAqG18Mjv/IKEArFvxYKmde7w9Vw3/kwbHS+cB4q\ni8hgAM+palWKv88CMAsAKioqanYnS5+IClBJCdDW1vH14mKgtTX99THczV85ESrbUNV6VR2nquMG\nDhzoujlEgZGsGHT2elcY7lLgCwIRJVdcnN7rXWG4S65vO/13AJsADBeRvSJyvcv2UG7zIxCdM8dc\nmhExP+fMyWw5L82ald7rXWG4S857H6fzYE9lSsWP3q6zZ8evL/KYPbt7y3lt1SrV4uL4bRYXZ/aZ\nOQx1fgJ7KlMh8SMQtQ1tvQ53bTEEJlt5EyoT2fAjELUNbb0Od20xBCavsSBQXvAjELUNbb0Od20x\nBCavsSBQXvAjELUNbdMJd70Mvjm8NHnOJmgIyoOhMnXGj0B09uxocFtcnDwoXrVKtaQkPtwtKem4\nfT+Cbw4vTTbAUJkoO2zDXc5tTK4wVCbKEttwl3MbU9CxIBBlyDbc5dzGFHQsCFRwvA5YbcNdzm1M\ngWcTNATlwVCZMuVXwGobaHNuY3IBDJWJOmLAmhr3Tf5iqEyUBAPW1LhvKGVBEJEzReRVEflQROpF\npH/M3/5vdppH5C0GrKlx31BnZwjLASwEcCaAnQA2isjp4b+V+twuIl/4Fey6GP7aaxz+mjorCH1U\n9XlVbVHV+wH8CMDzInIugNwJHohi+DF/75w5wPLl0cHs2trM81wrCpzbmFKGyiLyBoALVXV/zGtj\nADwJ4ARVHZCdJkYxVKYgcjX8NZEtL0LlXwIYGfuCqr4J4GIAT2XWPKL84Wr4ayKvlaT6g6r+LsXr\newDc6FuLiHJMcXHqMwSiXMLbToky5Gr4az/WR4Ut5RkCEdlZtsz8rK83ZwrFxaYYRF6PaGgwrx86\nZJ7v3h0tGt0Jbr1eHxF7KhNlidc9gdmzmGx51lNZRIaJyAsisi38fIyI3O1FI4kKidc9gdmzmLxm\nkyE8AuBOAMeA9juNrvazUURxEs9ic+isNpbXPYHZs5i8ZlMQylU1cagK3l1NKU2aZDo2RR6TJiVf\nzioQXbgQuPXWaBFQNc8XLvSl7d1qo6VFi4AePeJf69Gj+z2B/ehZzJAawL59wIYNwCOPAPPmAZde\nCsyf77pVWWETKu8LD1mhACAi3wbwsa+topw1aRLwwgvxr73wgnl9/froa1aBqCrQ0gIsWWKeL15s\nisGSJUBdnfm7iG+fxY/Q1suTnUgb7rrLXCaqqDDFoLttK6iQOhQyH3D7dmDHjvifzc3R5Xr1AoYP\nB6qr3bU1i7oMlUXkKwDqAZwH4FMA7wOYoapJ4ix/MVQOvs6Oz7H/q1kHopEzgkhRAEwxWLzY12KQ\nVhsdrc9rQW9ftxw+DOza1fHA//bbwJEj0eUGDgRGjABGjoz/WVFhTpdynG2o3GlBEJEiAN9W1cdF\npDeAIlU94GE708KCEHy2BaGoKPm3YxHz5a3DG2P/UYZCvhcDIM02Olif14Levk41N3f8pr9jB/D+\n+9EPJQIMGZL8wD8g6yPxZJVtQej0kpGqhkTkRwAeV9WDnrWOCl5FRfJvox0C0cgZQqxbb83KGYJ1\nGx2tD0DHy2YZXEbzpX1eCoXMtbHIwT72wN/UFF2urMxc5hk/Hpg5M3rgHzrUXAKi1LqaUg3ATwHM\nB3AagBMiD5vp2Lx+cApNt2ymYbz44vgpGCOPiy/uuK4up2sMhVTr6lQBfbRvnQpC+mhf81zr6szf\nfbRqlWppaXwbS0u7P6Wk51NULlgQvx8i+2vBgmC0r7sOH1Z9803Vxx5T/ed/Vr36atWxY1V79Ypv\n3Iknqp5/vuqNN6r+67+qrl6t+t57qm1tWW5w8MFyCk2bgvB+ksd7Niv3+sGC4E46B4vEopBYDGLX\n2VWBeWPaAn2opE6BUHh9IX2opE7fmLbAw0+Xun09esR/lh49MjtAejanckyxbC8Kic9dts9Gc7Pq\nK6+orlypOn++6mWXqZ5+umpRUXSHi6gOGaI6darqT36iWl+vumGDalOTjw3LP7YFgT2VyYqrwNFs\nVwHEXgZRVFaK70Fn4ENWdRe4WwuFgA8/TH59/5NPosv17Gku8yRe3x86tOO9tZQ2T0Ll8IpmJntd\nVX/bzbZ1GwuCO64CR5dBZ06ErOomcO/g6NHUd/NE7mMFgBNOMAf6yEE/cuCvrOTwsD7yJFQOGx/z\nexnMfAhbAGRcEERkCoAlAIoBrFTVX2S6TvKHq8DRZdBptt3x7KSiIiDfviNnCLEyDdy1i5D600+T\nf9t/7734Kjl4sDnYT5wY/61/4MDutYuyw+a6UuwDQD8Af0j3fUnWUwzgXQBfAdADwBsARnX2HmYI\n/rC5buwqcExnu15f/35j2gJdWhSfXywtyk5+0SU/MoRISN3Wprp7t+rzz6teeKFqTY1qba3qySfH\n/4fo2VP1zDNVp09Xvece1d/9TvX111UPHvT601KG4FWo3OENQCmA7em+L8l6vgpgbczzOwHc2dl7\nWBC85/KAm04bs16wQiHdfok5wC6GKQqLYZ5vv8T/O5ysZHqX0dGjqtu2qT7xhOrPfqY6fLi230qV\neOA/7zzV665Tve8+1WefVX3nHdXWVr8+GXnMtiDYZAjPIjxsBczYR6MA/F5Vb8/kzCQ8BMYUVb0h\n/Pz7ACao6o9SvYcZgvcCH5xa8uNzDK5UzN1zK+YiGto+iDo8WLEYH+wO0GWjrvohtLSkvswTO9Vb\nZaX5Gbsjb7wRWLEiL3rrFjIvQ+XamKetAHar6t4M2wcRmQ7gkoSCcI6q/jhhuVkAZgFARUVFze5k\n/+qp23IiOLXgx+cw61RozBiQghBEJHj7RhXYuzf5gf/vf48u16MHMGxYx7t5hg0DevcOTkhNnvIy\nVL408WxARH6Z6RkCgL0wnd0iBgH4KHEhVa2HGUsJ48aNy517ZHOE096pNt9uLfnxOSpOM2cIsRbj\nVjx42mLEB81Z9MUXwDvvJD/wH4wZTOD4482BfupU83P4cGDUKHMqVVKSfF+ru17hFBBdXVMCsCXJ\na2/aXI/qYr0lAN4DMATRUHl0Z+9hhuA9Z71T07j+7XWGYJWFuM4QWlpUX31V9Te/Ub39dtVvflN1\n2DDV4uL4D3naaaqTJ6vecovq8uWqf/6z6t//Ht8+m33tU0c3CgZkGioDmA3gPwAcBPBmzON9AKts\nVt7lxoFLAeyEudvorq6WZ0HwR9bD4jQOPl4f6NMqgAsW6PZL6rSyImTWWREuEt0cGiLpfvjwQ9U/\n/Ul16VLVOXNUL7pI9dRT4xtYWqo6erTqlVeq3n23aezmzaoHDthtw/ZA7/FQGBQctgUhZYYgIv0A\n9Afw3wHcEfOnA6r6D6/OUNLBUDmPRC5PdNHL1vkQ1F5c1jp2LPVlns8/jy7Xr1/HUThHjjQjdJbY\nXN1NwXJfJ/18GVzGo+DwLFSOWeFJMB3TAACqmvWZW1kQ8oxFgJlTQ1B/9lnHUTi3bwfefRdojZlk\ncNCg5Af+k0/27+DLsLigeRYqi8h/AfAAgC8B+ARAJYDtAEZn2kgqYJYBZlphcSjU8aCXcLtk2uFz\n4jfkUAj4+OPkB/6PYu6JKCkx4/CMHg1ceWX0wD98ONC3b4qNdYPNN3rLfe35din3dHVNCSbsHQDg\n9fDziwDU21yP8vrBDCFP+JEh1NaaIZIjQx+3tZnntbVxi1mv74svVG++WfUb31BdtEh15kzV8eM7\nDn963HGqEyao/uAHqr/4heozz6i+/bZ5v99chcXMGnIOLDMEmwuTx1S1WUSKRKRIVf8sIr/0rUJR\n/hMxt0XGXsdevNj87fjj475pWs0bHAoB+/cDW7cCNTXA5s3m59atwNixcWcKiesbOegA7r9hB6aG\ndgD/Lebb/jvvRC/zPPcc8OUvm3V88YUZn+fuu81tnKec4uabsardfNNp7GtPt0s5yaZj2noA3wLw\nC5gzhU8AjFfV8/xvXjxmCHnGy8sOoVC0CESMHWuKg4jpnJVsQvX//M/o8iUlwBlnxI/EuXo18Nhj\n0WWCNLy0q7A4ne1SIHjZU7k3gMMww1bMgBncrkFVm71oaDpYEKhTX3xhxtWPmDnTDL+8Y4c5g4jo\n2zf5vLqnnw6UlsavM+hhrKv2BX2/UBzPQmVVPSgilQCGqur/EpFymJFKidz4/PPkoe6OHfHL/e53\nQG0t8L3vxR/8Tz3V7uDlMox1FRbbcLVd8l9XIQOAGwG8BuDd8POhAF6wCSi8fjBULiChkOrHH6u+\n+KLqsmWqP/6x6te/rjpoUHyoW1xsRuns2VPbe+5u2qRaVWWen3xy9+bYdRnGBrlnMXs05yR4GCrf\nDOAcAH8NF5Bd4T4JRJlrbQXefz/59f3Yyzx9+phv+BddlPwyz2mnmTxg2jRgwgTga18Dtm0zuUB3\nvrW6CmNdhcW2XG2XssImQ/irqk4QkddV9WwRKYEZ32hMdpoYxQwhhx082PEyz44dZtrFL76ILnfq\nqR2nVxwxwtzl09nBRhWYOxdYujT62i23AA8+mNlBykUY6yosToer7VK3eBkq/w8ALQBmAvgxgDkA\n3lLVu7xoaDpYEAJO1Uycnuzb/ocfRpcrLjbf7BND3REjzPANmWw/6EGnbRtz4bNQzvBy+Os7AFwP\nM9DdTQBWA1iZWfMoI15/O0s36GxtNQP/vPWWuYsn9sDf0hJdvndvc5CvrY0/8J9xhhmX35ZFD+S0\ngk4/vt16GQIztCVXUoULACpsQohsPjINlV1NAekpr3uJdra+zz9X3bJFtaHBzK17xhkmrE3srXvK\nKaoTJ6rOnq26ZInqunWqe/Z4EzDa9EB2PaKnlyEwQ1vyATwY/npLzO9P2qzM70cmBcHZuP9e8vpg\nEQqZcfQBM7Ty8uXmYAuo9u0bv7NEzM+vfEX1tttUJ00yz2+6yb+DVOTgD0SLQuLzCFd35vhRjDg0\nBHnMi4LwerLfXT4yKQiVlfHHt8ijsrLbq3Qj9oATedgczFpbzcTozz5rJkq/7jozcXr//h13ykkn\nqc6Yofrzn5sJ2LdtUz18uHvbzVRsEYg8EotBRGJbkrWtu/uvM+ms06aN6SxHZMG2IHQ2H8IWVa1O\n/N2lTELlfJk7GEDngeOhQ8DOnR2D3Z07gaNHo+85+eTodf3hw+OvWQct6AyFTBAd0daW2aTvfnwO\nV/uGyIJtqNzZv6qzROQzETkAYEz4989E5ICIfOZdU7Mj1fDGWZk72EuaJHCsqTFz5w4ZYu7XP/ts\n4LvfBf7lX4DXXzezv9xyC7ByJfCXvwD/+IcZ2+ell4BlyzrODHPrrR2rZ7LtJlvOa5EximLV1HS/\niqf7OZLth0zXadvOrrabznJENmxOI4LyKLgMobVV9d13VZ97TvX++81lnsTpFSOPgQNVv/td1Z/9\nTPX3vzcB780353bQmU6GYCPdzxH04aWZNZAleNhTOS9YDaPsyuHDqS/zHDkSXe6kk4BevYCqKuC6\n68zwy8OHm9sR+/cHFi40y6kCGzeajk0lJZn3dnXVO7WoyPRLiIxaWlQUHdq6X7/0Lxul8zlUgz28\ntO1yROmwqRpBeeT8WEZNTaobNqjW16v+5CeqU6eqDhkSvYMHUC0qUj39dNXLLlOdP1915UrVV15R\nbW6Orsfr8DToQWfimUB3xiaKlc7n9Xof2rbPZrt+BOSUl5BpqBxEOdFTORQyczQmG4lz377ocr16\nmW/3icMwDx0KlJWlXn86lEFnxlztQ9vt8r8xWfCypzIlc/iwGYcn8aD/9tvxl3kGDjQH+yuuiD/4\nV1RkdqdMVzRF0MnervY9ldPZh7brtG0fezSTCzanEUF5OLlktG+f6saNqo88ojpvnuqll5rOWbGX\neUTMa5deapZ55BHznn37st9eVfZ27YxtEOuq93PQg37KSWConIZQyCTNyS7zNDVFlysrM5d5xo83\ns3HFXubp1ctd+xNxiOLkNI0g1nYfprNOG0EP+imvMUP44x+B6dPNJaCIAQM6jsQ5cqS5zFOcQ5PF\neXkZI19ELrPYzgdssw/TXadtO72aWY0KnmfDXweJLwVh505gxYr4A/+JJ3q7DQoWP4JYhrsUYF70\nVC4Mw4YBDzwA3HgjcMEFhVkMEr8UpPqSkNg7ONMxP2y366VUQWwm2/ZjnUQOsCAUuoUL4w9ekYNb\npJNbxMSJ8UNGRIaUmDjR3+16KfbSTl2d+Qx1deZ5dw/gfqyTyBEWhEIWG4hGDl6Rg1tLS/RgFgqZ\n+Y23bo0WhZoa83z//vTPFGy367VUQWxdXfeDWD/WSeQIM4RCZxuIxhaBiNghJfzarh/8CGIZ7lKA\nMVQme7aBaC4MQ01EHTBUJjuqwNy58a/Nndvxsk0oBFQnTIlRXZ38cpFNWMwglihwnBQEEZkuIn8T\nkZCIdFm1yCeqwFe/CixdauZLCIXMz6VLzeuxGcKXvgS88QZw1lnmzOCss8zzL30pvijYhMUMYokC\nydUZwjYAVwB42dH2KR0iZhhtAKitNc9ra83zkpLkvXY7C4sZxBIFk834Fn49ALwEYJzt8jk//HUQ\nhUKqt9wSHZcJMM+TDbVsu5yLIaOJKCXkwvDXIvISgPmqapUUM1T2iW246/VyRJQVzkNlEVkvItuS\nPL6Z5npmiUijiDQ2xQ40R12z6VlsG+6ms5xNSJ0O2x7NLno+E+UTm9MIvx7gJSP/1NbGzzscmY+4\ntja6jNdDLYdCqhMmxF9OilxmmjCB8wsTOQLLS0a87TQf2fYstg13XYXAahlS2y5HRJ2zqRpePwBM\nA7AXwFEA/w/AWpv38QwhDZEzgthwN/aMIZaXcyrbhs+2OL8wUcaQC6Fyuhgqp8nrnsW21ONQ2XZ9\nXm+XKE84D5UpRmLRzUYRjlwmihU7Wmln7cmkfepxqBy5/BMrk9CbiFJiQfCbi2GeYzODsWPNmcHY\nsfGZgh/tU8uez+msz6ZHs+1RMFx2AAAIEklEQVRyRNQpzqnsp9iwE8h8vl1bRUVAv37xo5Fu3myK\nQb9+0csqrtpni/MLE2WXTdAQlEdOhsouw87EADlVoOxl+7wOlSPr7Ox5ussRFRgwVA6QoIedrkJg\nIsoKhspB4TLstAmLvW4fw12inMWC4CeXYaeLYagZ7hLlNIbKfnIVdtqGxV63j+EuUU5jhpANiXfr\nZOPundhv6xGp5iz2un0uPi8RpcQ5lYnhLhEBYKhMXvcYJqK8x4KQj7zuMUxEBYEFgYiIALAg5CcR\nYNOm6FlBUVH0bGHTJuYIRJQUQ+V8xlCZiMBQmdLpMexieG4iChwWhHyUTo9hF8NzE1EgsadyPrLt\nMRz04a+JKKuYIeQzmx7D6fRoJqKcxJ7KZI/hM1FeY6iciUIKWRk+E1EYC0KiQgpZGT4TUQwWhFix\nIWvk4Bc5YLa05N834lThc11d6vC5EPYLUYFihpCoEENWhs9EeY2hciYYsibH/UKUkxgqdxfnBE6O\n+4Uo77EgxOKcwMlxvxAVBPZUjsU5gZPjfiEqCMwQkuGcwMlxvxDlJGYImUg8yPGgZ3C/EOW1wioI\ntj1t2SOXiApQ4RQE25627JFLRAXKSUEQkftEZIeIvCkiT4vI8b5u0LanLXvkElEBcxIqi8hkAC+q\naquI/BIAVPX2rt6XUahs29OWPXKJKM/kTE9lEZkG4NuqOqOrZTO+y8i2py175BJRHsmlu4yuA7DG\n963Y9rRlj1wiKlC+FQQRWS8i25I8vhmzzF0AWgE0dLKeWSLSKCKNTU1N3WuMbU9b9sglogLmW09l\nVZ3U2d9F5AcAvgHgYu3kupWq1gOoB8wlo241xranLXvkElEBcxUqTwHwAIBaVbX+2u9JhmDT05Y9\ncokojwQ9Q3gIQF8AfxKRrSKyIitbte1pyx65RFSAnAxup6pnuNguERGlFoS7jIiIKABYEIiICAAL\nAhERhbEgEBERABaEzHCYbCLKIywI3cVhsokoz7AgdAeHySaiPOSkH0LOix3SYsmS6FDZHCabiHKY\n8+Gv05Hx0BVe4zDZRJQDgj50Re7jMNlElGdYELqDw2QTUR5ihtAdHCabiPIQM4RMcJhsIsoBzBCy\ngcNkE1EeYUEgIiIALAhERBTGgkBERABYEIiIKIwFgYiIALAgEBFRWE71QxCRJgC7fVj1iQD2+bDe\nXMR9YXA/GNwPRq7vh0pVHdjVQjlVEPwiIo02nTYKAfeFwf1gcD8YhbIfeMmIiIgAsCAQEVEYC4JR\n77oBAcJ9YXA/GNwPRkHsB2YIREQEgGcIREQUxoIQJiL3icgOEXlTRJ4WkeNdt8kFEZkuIn8TkZCI\n5P1dFYlEZIqIvC0i74jIHa7b44qI/JuIfCIi21y3xSUROU1E/iwi28P/Lupct8lPLAhRfwJQpapj\nAOwEcKfj9riyDcAVAF523ZBsE5FiAP8TwFQAowBcIyKj3LbKmd8AmOK6EQHQCmCeqo4EcC6Am/P5\n/wkWhDBVXaeqreGnrwIY5LI9rqjqdlV923U7HDkHwDuq+p6qfgHgfwP4puM2OaGqLwP4h+t2uKaq\nH6vqlvDvBwBsB/Blt63yDwtCctcBWOO6EZR1XwbwYczzvcjjf/yUHhEZDOBsAH912xL/FNScyiKy\nHsApSf50l6r+n/Ayd8GcJjZks23ZZLMfClSyKe94Gx5BRPoAeBLAXFX9zHV7/FJQBUFVJ3X2dxH5\nAYBvALhY8/h+3K72QwHbC+C0mOeDAHzkqC0UECJSClMMGlT1Kdft8RMvGYWJyBQAtwO4XFUPuW4P\nOfEagKEiMkREegC4GsAfHLeJHBIRAfAogO2q+oDr9viNBSHqIQB9AfxJRLaKyArXDXJBRKaJyF4A\nXwXwRxFZ67pN2RK+qeBHANbChIePq+rf3LbKDRH5dwCbAAwXkb0icr3rNjnyTwC+D+Br4ePCVhG5\n1HWj/MKeykREBIBnCEREFMaCQEREAFgQiIgojAWBiIgAsCAQEVEYCwIVDBFpi7l1cGt4KIJ013G8\niMzxvnXt6x8hIptE5KiIzPdrO0TJ8LZTKhgi8rmq9slwHYMBPKeqVWm+r1hV2yyWOwlAJYBvAfhU\nVe/vTjuJuoNnCFTQRKQ4PBfGa+G5MG4Kv95HRF4QkS0i8h8iEhn19BcATg+fYdwnIhNF5LmY9T0k\nIteGf/9ARO4RkY0ApovI6SLyvIhsFpENIjIisT2q+omqvgbgmO8fnihBQY1lRAWvl4hsDf/+vqpO\nA3A9gP2qOl5EegJ4RUTWwYx6Ok1VPxOREwG8KiJ/AHAHzLwZYwFARCZ2sc0jqnp+eNkXAPxXVd0l\nIhMALAPwNa8/JFF3sSBQITkcOZDHmAxgjIh8O/y8H4ChMAPd3SsiFwIIwQyDfXI3tvkY0D5a5nkA\nfm+GxwEA9OzG+oh8w4JAhU4A/FhV48ZsCl/2GQigRlWPicgHAMqSvL8V8ZdeE5c5GP5ZBKAlSUEi\nCgxmCFTo1gKYHR7iGCIyTER6w5wpfBIuBhfBBL0AcABmEMSI3QBGiUhPEekH4OJkGwmPof++iEwP\nb0dE5Cx/PhJR9/AMgQrdSgCDAWwJD3XcBHOHTwOAZ0WkEcBWADsAQFWbReSV8OTza1T1NhF5HMCb\nAHYBeL2Tbc0AsFxE7gZQCjNF5xuxC4jIKQAaARwHICQicwGMyudJWSg4eNspEREB4CUjIiIKY0Eg\nIiIALAhERBTGgkBERABYEIiIKIwFgYiIALAgEBFRGAsCEREBAP4/R0SKooMt3SQAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa262cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 计算分类正确率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.93\n"
     ]
    }
   ],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)\n",
    "loc_n = np.where(s < 0)[0]\n",
    "y_pred[loc_n] = -1\n",
    "accuracy = len(np.where(y == y_pred)[0]) / len(y)\n",
    "print('accuracy: %.2f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 注意："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过实例验证，优化后的PCA算法不一定每次都能找到最优分类线。在足够的迭代次数下，使用一般的PCA算法，竟一般都能得到最佳的分类线，即使开始的时候，分类线会有所振荡。但最终都基本能寻找到最佳位置附近。这也是PCA算法的特点之一。下面是使用一般PCA算法处理非线性可分样本实例的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XucVVX9//HXZ2ZAGO4C5gW56M8b\nIhCMad7gm4a31MwslSJTQ1ET0SxNE7IwK78SSOI10yTT8pqJ+EXT0NQcCNFERQ1wKB9cFFQQFObz\n+2Odw9wOzD4z+8w+l/fz8TiP4ezZs/c656H7s/dan/VZ5u6IiIiUJd0AERHJDwoIIiICKCCIiEiK\nAoKIiAAKCCIikqKAICIigAKCiIikKCCIiAiggCAiIikVSTcgG7169fL+/fsn3QwRkYIyb968Ve7e\nu7n9Ciog9O/fn+rq6qSbISJSUMxsaZT91GUkIiKAAoKIiKQoIIiICFBgYwiZfPrpp9TU1LBhw4ak\nm1KQOnToQJ8+fWjXrl3STRGRhBV8QKipqaFLly70798fM0u6OQXF3Vm9ejU1NTUMGDAg6eaISMIK\nvstow4YN9OzZU8GgBcyMnj176ulKRIAiCAiAgkEr6LsTkbSiCAit8umn8M47oLtkESlxCggffQQr\nVsArr8DixbB2LWS5znR5eTlDhw5l0KBBnHzyyaxfvz7rZpx11lm8+uqrAFx99dUNfnfQQQdlfTwR\nkWwpIPToAfvtBzvvDOvXh6Dwyivw7ruwaVOkQ3Ts2JEFCxbwyiuv0L59e2688casm3HrrbcycOBA\noGlA+Pvf/5718UREslVyAWHmTOjfH8rKws+ZM4H27UNA2G8/2G03aNcOampg4UJYsiQEiogOPfRQ\n3nzzTQCuu+46Bg0axKBBg/jVr34FwLp16zj22GMZMmQIgwYN4p577gFg5MiRVFdXc+mll/Lxxx8z\ndOhQRo8eDUDnzp0B+PrXv86jjz665Vynn3469913H5s3b+aSSy5h//33Z/Dgwdx0002t/p5EpPQU\nfNppNmbOhLFj667vS5eG9wCjRxOixPbbh9f69aEr6b33YNUq6NwZdtgBuncP+2WwadMmZs2axVFH\nHcW8efO4/fbbeeGFF3B3DjjgAEaMGMHbb7/NzjvvzF/+8hcA1q5d2+AY11xzDdOnT2fBggVNjn/K\nKadwzz33cMwxx/DJJ5/wxBNPMGPGDG677Ta6devGiy++yMaNGzn44IMZNWqUUklFJCuJPiGYWXcz\n+5OZvWZmi8zs87k83+WXN73ZX78+bG+isjI8QgweDH36hMHnt9+Gl1+G//wnvE9J39FXVVXRt29f\nzjzzTJ555hlOPPFEOnXqROfOnfnKV77C3Llz2W+//ZgzZw4/+MEPmDt3Lt26dYvc/qOPPponn3yS\njRs3MmvWLA477DA6duzI448/zp133snQoUM54IADWL16NYsXL27ZlyQiJSvpJ4SpwGPu/lUzaw9U\n5vJky5Zltx2AigrYcUf4zGfCgPOKFSEg/Pe/Yfxhhx22jCHU51sZmN5zzz2ZN28ejz76KJdddhmj\nRo3iyiuvjNT+Dh06MHLkSGbPns0999zDqaeeuuVc119/PUceeWSk44iIZJLYE4KZdQUOA24DcPdP\n3H1NLs/Zt2922xswC91Fe+4JgwaF7qO1a+G116C2NnQr1dZu2f2www7jwQcfZP369axbt44HHniA\nQw89lP/85z9UVlbyjW98g+9973vMnz+/yanatWvHp/WeQOo75ZRTuP3225k7d+6WAHDkkUcyY8aM\nLX/zxhtvsG7duggfSkSkTpJPCLsBK4HbzWwIMA8Y7+45u5JNntxwDAFCz9DkyVkeqEMH2HXXMBD9\n3nth25IlYSC6Z0/YYQeGDRvG6aefzuc+9zkgpJV+9rOfZfbs2VxyySWUlZXRrl07ZsyY0eTwY8eO\nZfDgwQwbNoyZM2c2+N2oUaMYM2YMxx9/PO3bt99y7CVLljBs2DDcnd69e/Pggw9m+aFEpNTZ1ro2\ncn5isyrgeeBgd3/BzKYCH7j7jxrtNxYYC9C3b9/hS5c2XOdh0aJF7LPPPpHPO3NmGDNYtiw8GUye\nnBpQbg33uvkM778ftnXrFp4iunYNTxd5LNvvUEQKi5nNc/eq5vZL8gmhBqhx9xdS7/8EXNp4J3e/\nGbgZoKqqqtXRa/ToGAJAY2bQpUt4ffIJrFwZXosXw3bbhcDQs2cYjxARyVOJXaHc/V0ze8fM9nL3\n14HDgVeTak9s2reHXXaBnXYKTwsrVoTSGMuXb+lOomPHpFspItJE0res3wVmpjKM3ga+nXB74lNW\nFgJAz56wbl14Yli1Kvzs0gV6997mnAYRkbaWaEBw9wVAs/1aBa9Tp/Dq0ycEhRUrwpyGdu1CYOjd\nO/xbRCRBST8hlJYIcxro1CnvB6FFpDgpICQhPaehe/dQdnvFivDk8N57IQ92hx1C+Qx1J4lIG9IV\nJwZmxsUXX7zl/bXXXsukSZOi/XGHDiH/dciQ8LO2NsxpWLgwzGvYuHHLriqLLSK5VHoBofG8ixjm\nYWy33Xbcf//9rFq1quUHKS8PTwb77htmQ3fuHEpwv/wyvPkmfPCBymKLSE6VVkCYNAkmTKgLAu7h\nfdS7+a2oqKhg7NixTJkypcnvVq5cyUknncT+++/P/vvvz7PPPrtl+xe/+EWGDRvG2WefTb9+/UJA\nMePLY8Yw/OtfZ98xY7j5iSfgo4+49LvfDUX0Bg1i9GmnASqLLSIxc/eCeQ0fPtwbe/XVV5tsy6i2\n1n38eHcIPzO9b6FOnTr52rVrvV+/fr5mzRr/5S9/6RMnTnR391NPPdXnzp3r7u5Lly71vffe293d\nzzvvPL/66qvd3X3WrFkO+MqVK93dffXq1e7uvn79et9333191YoV7qtWeaeOHd1ffNF93jz3JUu8\nU6dO7u5+//33+5gxY9zdfePGjd6nTx9fv36933TTTf6Tn/zE3d03bNjgw4cP97fffrtJ+yN/hyJS\nkIBqj3CNLZ1BZTNI38FPnRpeAOPHh+2tzOzp2rUrY8aMYdq0aXSsN/Fszpw5W5bGBPjggw/48MMP\neeaZZ3jggQcAOOqoo+jRo8eWfaZNm7bld++88w6L33qLngceGAaZ99mnbhC6thZef52jDzyQCy64\ngI0bN/LYY481KIu9cOFC/vSnPwFh7YXFixdrnQQRyah0AgLUBYV0MIBYgkHahRdeyLBhw/j2t+vm\n19XW1vLcc881CBKw9fLYTz31FHPmzOG5556jsrKSkSNHsmHDhrodOnWCAQPCnAYz2LiRDsuXM3LI\nEGbffTf3PPYYp6Zqc7jKYotIFkprDCE9ZlBf/TGFVtp+++352te+xm233bZl26hRo5g+ffqW9+l1\nEw455BDuvfdeAB5//HHeTxXFW7t2LT169KCyspLXXnuN559/fsvfNiiL3a5dCAj77Qe7784pxx3H\n7XfcwdynnuLIPfeEdetUFltEslI6ASEdDKZODd1EtbXh59SpsQaFiy++uEG20bRp06iurmbw4MEM\nHDiQG2+8EYCJEyfy+OOPM2zYMGbNmsVOO+1Ely5dOOqoo9i0aRODBw/mRz/6EQceeOCWY6XLYo+u\nX53PDHr0YNQZZ/C3hQs54rDDaL9uHSxaxFkHH8zAAQMYNmwYgwYN4uyzz2bTpk2xfE4RKT6Jlb9u\niaqqKq+urm6wLavSzZMmwZo1dd1E6SDRvXurM42ytXHjRsrLy6moqOC5555j3LhxGddRbpHNm2H1\n6jDWsGFDmCGdLpGRWkOhPpW/FiluhVD+uu1NmhSCQHrMID2mkECpiGXLlvG1r32N2tpa2rdvzy23\n3BLfwdNzGnr3hg8/DIHhv/8Nr+7dw++6dFGJDBFpoLQCAjS9CCZ0Udxjjz345z//mduTmIUFerp2\nDTOe0xVX16wJM6TT6zSIiFAkAcHdMd3tbtt224XMpPSynytWwLJl+DvvwMcfw+uvw157Jd1KEUlQ\nwQ8qd+jQgdWrV281jVMaKSuDXr1gn33wvfZidYcOdFiwAPbeG0aNgocfDmMQIlJyCv4JoU+fPtTU\n1LBy5cqkm1KQOlRW0ucrXwnluGfMgBNOgH794Nxz4cwz1aUkUkIKPstIYrRpEzz0EEyfDk89FbqZ\nTj0Vzj8fhg9PunUi0kJRs4wKvstIYlRRASedBH/9a6iyesYZ8Mc/QlUVfP7zMHNmg3LcIlJcFBAk\ns0GD4IYbYPnyMHlv9Wr4xjfCmg0/+lFYq0FEiooCgmxbt25wwQXw2mswezYccABMngz9+8PJJ8PT\nT8c2y1tEkqWAINGUldVlIb31Flx0ETzxBIwcCYMHw003wUcfJd1KEWkFBQTJ3oAB8ItfhG6j224L\nhfbOOSfMc5gwARYvTrqFItICCgjScpWVYeB53jz4+9/h2GPh178OS4AedRQ88ojmNIgUEAUEaT2z\nuiykZcvgqqtCltJxx8Eee8C114bZ0SKS1xQQJF477hiykJYsgXvvhV13hUsugV12gbPOglzXbxKR\nFlNAkNxo164uC+mll2DMGLj7bhg2DA45BP7wB/jkk6RbKSL1KCBI7qWzkGpq4Lrr4N13wwzofv1C\nSfL//CfpFooICgjSlnr0CFlIb7wBs2aFchhXXRUCwymnwDPPaE6DSIIUEKTtlZXVZSEtXhyWMp09\nGw49FIYOhVtuAa39LNLmFBAkWbvvHrKQli8PgcAMxo4NcxouvjhMghORNqGAIPmhsrIuC2nuXDjy\nSJg2LaStHnts6GKqrU26lSJFTQFB8otZXRbS0qVw5ZUwfz4cc0yY8DZlCrz/ftKtFClKCgiSv3be\nOWQhLV0aUlZ33DHUUOrTB84+GxYuTLqFIkVFAUHyX/v2dVlI//wnnHYa/O53MGQIHHZYmAD36adJ\nt1Kk4CkgSGFJZyHV1ITB6Joa+PrXQznuq64KcxxEpEUUEKQwbb99yEJavDikrw4eDBMnhgV8Tjst\nFNvTnAaRrCQeEMys3Mz+aWaPJN0WKUDl5XVZSG+8AeedB48+CgcfHCa+/eY38PHHSbdSpCAkHhCA\n8cCipBshRWCPPUIWUk0N3HhjGFc488wwCP3978O//510C0XyWqIBwcz6AMcCtybZDikynTvXZSE9\n/TQcfnioobT77nD88WFWtOY0iDSR9BPCr4DvA/q/U+JnVpeFtGQJXHEFvPBCKJux994wdSqsXZt0\nK0XyRmIBwcy+BKxw93nN7DfWzKrNrHrlypVt1DopOn36hCykZcvCQj69esGFF4Z1GsaNg1deSbqF\nIokzTygTw8x+BnwT2AR0ALoC97v7N7b2N1VVVV5dXd1GLZSiN29eWPLz97+HjRth5Eg4/3w44QSo\nqEi6dSKxMbN57l7V3H6JPSG4+2Xu3sfd+wOnAE9uKxiIxC6dhbR8Ofz852HQ+atfhQEDYPJkWLEi\n6RaKtKmkxxBEktezZ8hCeusteOgh2GefMN6w667wzW+GcQfNaZASkBcBwd2fcvcvJd0OKXHl5SEL\n6fHH4bXX4JxzQoA48ED43Ofgjjtgw4akWymSM3kREETyzl57hSyk5cvhhhtg/Xo4/fQwOH3ppaHg\nnkiRUUAQ2ZYuXeqykJ58EkaMgF/+EnbbDb78ZZgzR91JUjQUEESiMIP/+R+4774w+HzppfDss/DF\nL8LAgTB9OnzwQdKtFGkVBQSRbPXtG7KQ3nkH7rwTunaF7343zGk4/3xYpEosUpgUEERaqkOHuiyk\nf/wDTjoJbr01PDEccQQ8+CBs2pR0K0UiU0AQicP++8NvfxueGn72s1B59cQTQ/2kn/0MNMteCoAC\ngkicevcO4wtvvw0PPBAqsP7whyE76VvfghdfTLqFIlulgCCSCxUVdVlI//oXfOc7cP/9YT7DAQeE\nJUA3bky6lSINKCCI5Fo6C2n5crj++lBhdcyYMBP68stDN5NIHlBAEGkrXbvWZSHNmRNWdbvmmrAe\n9EknwV//qjkNkigFBJG2ZhYW7XnggTDW8P3vh4V8vvAFGDQozIz+8MOkWyklSAFBJEn9+oUspJqa\nkKXUsWNYF3qXXeCCC+D115NuoZQQBQSRfNChQ10W0vPPhzUZbroprOw2ahQ8/DBs3px0K6XIKSCI\n5BOzuiykZcvgpz+FV18NAWL33eEXv4DVq5NupRQpBQSRfPWZz4QspCVL4E9/Cgv3/OAHYU7DGWfA\n/PlJt1CKjAKCSL6rqKjLQnr55VCG+957w4pvBx0UlgD95JOkWylFQAFBpJAMGgQzZoQ5DVOnwqpV\nMHp0KLh35ZVhu0gLKSCIFKJu3UIW0muvwezZYQb0T38aspZOPjmksWpOg2RJAUGkkJWV1WUhvfUW\nXHQRPPEEjBwJgweHTKWPPkq6lVIgthoQzGw/M3vezN4xs5vNrEe93/2jbZonIpENGBCykGpq4Lbb\nwtjDOeeEQegJE2Dx4qRbKHluW08IM4BJwH7AG8AzZrZ76nftctwuEWmpysq6LKRnn4Vjjgm1lPbc\nE44+Gv7yF81pkIy2FRA6u/tj7r7G3a8FzgceM7MDAXVOiuQ7s7ospGXL4Mc/hoUL4UtfCsHhf/8X\n3nsv6VZKHtlWQDAz65Z+4+5/BU4Cfgf0y3XDRCRGO+0UspCWLAkpq336wPe+F36edRYsWJB0CyUP\nbCsg/BzYp/4Gd18IHA7cn8tGiUiOtGtXl4X00kthCdC774bPfhYOOQT+8AfNaShh5gWUmlZVVeXV\n1dVJN0OkuLz/fiis9+tfh0ylHXeEs8+GsWNh552Tbp3EwMzmuXtVc/sp7VSk1PXoEbKQ3ngDHn0U\nhg2Dq64KcxpOOQWeeUZzGkqEAoKIBGVldVlIixfD+PFh0tuhh4YupVtvhfXrk26l5JACgog0tfvu\ncO21oRTGLbeEJ4TvfCes03DxxaFrSYpOswHBzPY0syfM7JXU+8FmdkXumyYiiausrMtCmjsXjjwS\npk2DPfaAY4+FWbOgtjbpVuaNmTPDiqhlZeHnzJlJtyg7UZ4QbgEuAz6FLZlGp+SyUSKSZ8zqspCW\nLg0prPPnh0lve+0FU6bAmjVJtzJRM2eGcfilS8MD1dKl4X0hBYUoAaHS3RuXqtiUi8aISAHYeWeY\nNClc8e6+O6zbcNFFoTvp7LPD5LcSdPnlTYdY1q8P2wtFlICwKlWywgHM7KvAf3PaKhHJf+3b12Uh\nzZ8Pp54Kd94JQ4bAiBHwxz/Cp58m3co2s2xZdtvzUZSAcB5wE7C3mS0HLgTOyWmrRKSwpLOQli8P\ng9HvvANf+1roSP/JT+Ddd5NuYc717Zvd9ny0zYBgZmVAlbsfAfQG9nb3Q9x9aZu0TkQKy/bbhyyk\nxYvhkUdCCe4rrwxXxdGj4bnninZOw+TJUF7ecFt5edheKLYZENy9llDUDndf5+4ftkmrRKSwlZfX\nZSG98Qacd16Y33DQQVBVBb/5DXz8cdKtjNWzzzYtIrt5c9heKJotXWFmPwI+Bu4B1qW3u3ubl0lU\n6QqRAvbRRyHlZvp0eOWV8DRx5pkwblxYy6HAVVRkripeXg6bEk7DibN0xRmEcYS/AfNSL12VRSQ7\nnTvXZSE99RR84Qtw3XVhEtzxx8Pjj+ftnIYo8wu2tsREIS090WxAcPcBGV67tfbEZrarmf3VzBaZ\n2b/MbHxrjykiBcCsLgtpyZKQl/nCC2HS2z77hIlva9cm3cotos4vaDx+0Nz2fBSly2hMpu3ufmer\nTmy2E7CTu883sy6EJ48vu/urW/sbdRmJFKmNG+G++0J30nPPQadOMGZMGHvYd99Em9a/fwgCjfXr\nF+JZ2rnnwowZTfcbNw5uuCFXrYsmzi6j/eu9DiUsq3l8q1oHuPt/3X1+6t8fAouAXVp7XBEpQNtt\nB6edBn//O1RXh5TV3/wGBg0KXUv33ZdYR3zU+QU33BAu/ukngvLy/AgG2ch6PYTUKmq/c/dWB4V6\nx+xPGKMY5O4fbG0/PSGIlJBVq0JQuOGGcIvepw+cc04osrfDDm3WjKhPCPksl+shrAf2aMHfZWRm\nnYH7gAszBQMzG2tm1WZWvXLlyrhOKyL5rlcv+P73Q2XVhx4K4wtXXAG77hpWenvhhTaZ0zB5cqjx\nV19lZWHNL4gqSrXTP5vZw6nXI8DrwMNxnNzM2hGCwUx3z7gsp7vf7O5V7l7Vu3fvOE4rIoWkvLwu\nC+m118JTwkMPwYEHwuc+B3fcARs25Oz0o0fDzTeHJwKz8PPmm8P2XGvr6qlRBpVH1Hu7CVjq7jWt\nPrGZAXcA77n7hVH+Rl1GIgLAhx/CXXeFQehXX4WePUNX0jnnhCt2EUhnN9UvmFdZ2bJgFGeX0THu\n/nTq9ay715jZz7NrTkYHA98EvmBmC1KvY2I4rogUqS13zN260P/n45h52Svw5JMhjfUXv4DddoMv\nfxnmzCn4EhlJVE+N8oQw392HNdq20N0H565ZmekJQaR0NXvHvGwZ3HRT2LBqFey9d0hbHTMGunZN\nrN0tVVaWOaaZZT9/r9VPCGY2zsxeBvYys4X1Xv8GSrPguYgkptk75r59w0jvO++EMtxdu8J3vxvW\naTj/fFi0qM3b3BpJVE/dVpfR74HjCAPIx9V7DXf3b+SuSSIiTUVeb6BDh7ospBdegK98JawLPXAg\nHHEEPPhg8sWFIkgiu2mrAcHd17r7Enc/NVXu+mPCIjmdzayAKnyLSDFo0R1zOguppgauvjpUXj3x\nxFA/6ZprII9T2ZPIboqSdnqcmS0G/g08DSwBZuWuSSIiTbXqjrl3b7jsMnj7bXjgAdhjj/B+113h\n9NPD7Og8NHp0mPxWWxt+5jrVNUqW0U+BA4E33H0AcDhQQBW+RSTfRcm3z+aOeavHq6ioy0L617/g\nrLNCWYz99w/zGu66K9RVyuHnyGvuvs0XUJ36+RJQlvr3P5r7u1y8hg8f7iJSXO66y72y0j3k1IRX\nZWXY3ibHW7vW/frr3ffaK+zcu7f7D3/ovmxZop8jTunreHOvKGmnc4AvA9cAPYEVwP7uflDuwlRm\nSjsVKT5x1wpq8fHc4YknwmS3P/85PIaccELIUBo5MrzPxXnbQJwT004g1C+6EHgMeIuQbSRS9Aq+\nC6AARM4eyvXxzOqykN56C773PXj66VBt9bjmL3lxf44kRFkgZx2wKzDS3e8AbgU+yXXDRJIWdWEU\naZ3tt89ue3Niyd/v3z9kIdXUwG9/G2k0N4l5A3GLkmX0HeBPwE2pTbsAD+ayUSL5IInSAdJ6sebv\nd+gA3/oWnHpq2543IVG6jM4j1B36AMDdFwNtV4xcJCHF0AWQllTXV5Tzvvde5r/d2vbmJFWdNMmq\nqHGpiLDPRnf/xFIDKmZWQZigJlLU+vbNPEhYSF0A0LQGULrrC3J7sYp63lx8z6NHJ3MhTuq8cYny\nhPC0mf0Q6GhmXwT+CPw5t80SSV4uugDivlOPcrykur6innfyZGjXruG2du0Kq6ulWEQJCJcCK4GX\ngbOBR4ErctkokXwQdxdA3IPUUY+XVNdXNudtnNHZTIan5MhW5yGYWV93z6veUs1DkEKWVL59Uvnx\n+d6+UhLHPIQtmURmdl8srRIpYUnl22fT9RVnl1bU8xbT4H2h21ZAqP/QtluuGyJS7OLOU298sd3a\n9qhdX3F3aUU9bzHk7xeLbXUZbVkpLdOqaUlQl5EUsjjXyIWw9nymlbPKymDz5uyPl1TXTdzfizQV\nR5fREDP7wMw+BAan/v2BmX1oZh/E11SReOQi1/7cc0OBTLPw89xzW75f3IPUW1tGMdvlFdOS6rop\nhvz9ohGlAl6+vFTtVLYmF5Umx41reLz0a9y4lu0Xt7KyzOctK2vZ8fr1y3y8fv3ibLUkgbiqneYT\ndRnJ1uSiu6OiInPXS3l5wxUYo+4Xt86dYd26pts7dYKPPsr+eOq6KV5xVjsVyXu56O7YWj984+1R\n94N4u7UaT/ra1va4F6CR4hSldIVI3stF+YPy8q3f+ddnFjpXGms8uSruEhJRP3M25y300gvSOnpC\nkKKQi1z79EWzue2dOmXer/H2uEtIRP3M2ZxX6z+UuCgDDfny0qCybMtdd4UBULPwM9OAcraDz+PG\nuZeXh/3KyzMPFJtlHow1a9l+cX/mqOfN5yUgpXXQoLJIU7kYfM73Eg353j7JPQ0qi2SQi8HnyZND\nplF9FRVNu26SWkBFJSQkKgUEKSm5KJPw7LNN00s3bQrb68v3hVtUQkLUZSQlJRe59knNQ4ib5iEU\nL3UZiWSQi7v0pOYhxH08zUMQPSGItFLUJ4S478B1Ry9R6QlBikfjm5Y8u4mJOl8h7nkISS2NKcVL\nAUFid8QRocsh/TriiMz7RerumDQJJkyoCwLu4f2kSTlpe0vaeMMNMG5c3Qzm8vLw/oYbGu6X1AI5\n2dDEtBIXZbJCvrw0MS3/HX545klQhx/ecL9Ik6Bqa93Hjw+/HD8+8/scinuiVs+emb+bnj1bdrxQ\nnbS20fFqW1ydVBPTihcRJ6YlfpHP5qWAkKwos2IzXfDSr/oil1qurfVFR45vsNOiI3MfDLJqY0Rx\nB4SXTpzo0yvG1wsKtT69Yry/dOLEFh1P5a+LV9SAoC4jiSTu5RWjdnfM/L0x/G9TGmwb/rcpzPx9\no8pxOZBp1u62tjfnvfey275N7gzuu4bzNk3lti4TMJzbukzgvE1TGdx3TYvGWTQxTRK94weOAl4H\n3gQubW5/PSEkJ+rdY9xPCP361voUGj4hTGG89+ub+yeEUMOoaZdMeXnLjhf7HXj9LrT0qxVdaXF3\nQUn+IN+fEMysHPg1cDQwEDjVzAYm1Z5SFmUgMerd4+GHZ96v8fZI5RTcmbBsAhcylV8xHqOWXzGe\nC5nKhGX1Bpqz+BzZuGLzJKYwAUifx5nCBK7YPKlFx4u9dIUZTGn49MSUKU3rbkf08LBJTK9o+Hmn\nV0zg4WGTWthAKThRokYuXsDngdn13l8GXLatv9ETQvyiDiRmc3fbeGC58YBy/XM3NyYxpdtEn0LD\nfvIpjPcp3Sa26HNEVlvrMzqM3/JEkj6vQ9jewrvwKJ85mzbG9oRQ71i3dRnvRq3f1qXtBvAlt8j3\nQWXgq8Ct9d5/E5i+rb9RQIj83uBpAAANpElEQVRf1At9TjJQGl9kMlx07rrLvbJjw26Myo61rQpY\nUfXcPnN3Vc/t8+DimG0GVoTvOu4uKMkfhRAQTs4QEK7PsN9YoBqo7tu3b06+rFKWTY3+WO9uJ05s\neLFJX4wmTmyya5xrEmTzOcIxaxscEGpbtX5BrKJ+h1l8117b8PMqGBSHQggI6jLKA4mkGmZxdxt3\nl1Y2TzpJDmhHFuVJIOqThJ4QilYhBIQK4G1gANAeeAnYd1t/o4AQv8QmI0W8+GR1oe+wueHn6LC5\n5V1L9eY/NB5DaKt5EJHE1RWUiy4oyRt5HxBCGzkGeAN4C7i8uf0VEHIj1q6gbETonojcFTRihK/u\nO9T77bo5fI5dN/vqvkPdR4xo2fGyOGZi4u4KykUXlOSFgggI2b4UEIpInE8Imze7Dx0afjF0aOb3\n2RyvcfsSKJvRrFx1BcXZBSV5QwFB8lcOxhB882b3IUMa7jhkSINgkNXxGrczrj71qF0tSXUFRf0M\nGmsoKAoIkt+y6HaI1KU1caL7BRc0vEhdcEHLj5cWZ9ZNUllBuejiUTZSQVFAKGZxD+jFedeai/M2\nustv8r62tmkwqB8UWjogmkRXS1JdQdnQE0LBUUDIILHB0zjFfbeX7wOJI0Y0HAdIjw/UH9jN1F20\ntW6jqJ8jmwtzS465rQtpUl1BUWgMoSApIDRSFLXe4/6fMRd3rXHKYrDYR4zIPIZQP3Bk+zmiXOhb\nkq4Zpaslqa6gKJRlVHAUEBopmlrvcXcTxHnXmgv1g0D61TgYpNuXaQyhtZ+jue6qbI6ZzX5RPkvU\n9uVC3N2HklMKCI1klX+e7+K+e4zzrjUXNm9ueN5MwSAXd+lxfofZPI0dcEDDIJAODgcc0LKuKil5\nUQNCySyQ07dvdtvzlntYU7i+CY1KQbvDmjUwdWrd7yZMCO/XrGm6b3PHy2a/uNXWwvDhDbcNHx62\np5lB9+4wfnxd+ecpU8L77t0bloPO5vPG+R1m08YosmmfSFRRoka+vDSGEHMGSjGNIaTb2dz7bJ8k\n4h7cjdqNl4vuLylZRHxCMC+gO4mqqiqvrq5u8d/PnAmXXx4WdenbNyxMMnp0jA1sC5MmhTvA9F1m\n+s6we/fwu/rcw2oxabW1Te9Eox4vm/PGaeRIWLsW5s0LnyX9xNCtGzz1VO6PF+U7jLuNUc+bzX5S\n0sxsnrtXNbtjlKiRLy/NQ0iJc7A46vGy2S9ucQ2c5sMTQrbtzMeBfik4aFC5hOWiG8M9/oyWJAJM\nSy62cXTPtaR9+diNJwUpakCoyPmzirS9rQ1gQtMBzKhdQXF3iyTVBZX+LqZOrduWaR3ibL7DqMeM\n2r4o582mfSJRRYka+fLSE0KW4hpkTXpwN07Z3s3H3T2XTTubO282+0lJQ11GEsmVV2ae+HXllU33\n69Wr4X69ejXdL+4yDnHKdX+/um4kT0UNCCUzD0EycA/dQAsWNNy+YEHY7t5wv1WrGu63alXT/aLm\nxtfv4khraTdLVHHPBcjVMUUSUlJppyWntrZpSmJZWdN9hg9vGBSGDq0bK0jbvBl23LFhUOjVC959\nF8rL67a5w4UXwrRpddsuuAB+9avME8Tq97vXv6jW5970b7eWghnnftnIxTFFYhI17VRPCMVq5MiG\nM3rTF/6RI+v2cYeLLsr8hHDRRXV39LW1UFWV+QmhqqrhrOEf/zhze+pvrx8Mxo8Pfz9+fMMni7RJ\nkxpuS/9t44HnqPtB5gHk1srFMUXamAJCMaqtresKSgeF9FPA2rV1F3CzkCU0dGjDvx86NGxPX9TK\nyqBr1/BEUF+vXmF7+knCHd5/v+HTAYT3779fd7GO2s0StQsqm64qEdm6KAMN+fIq2EHlJDJBolQJ\njTvnPVPJheZKL2zrfeNza5KWSIug0hV5Iql8ewhPAvX79zdvbjqGEHV+QTbzFdasgZdeqts2ZEjY\nT2UcRBKhMYR8kGRXRpQqoe6he6j+mEF6TGHo0KZ9+fUHfNPdPPWDgXu4+NcPBhDeDxnSss+b/s7q\ny6dqrCLFJMpjRL68CrLLKImujGwmksVddiGbLqMox1MZB5FWQ/MQ8kQS+fZlZWEQt1cvqK4O76ur\nw/v332/YrRJn+8ygR4+QZlrfBReE7dkeM+rgs+YCiMQjStTIl5eeEFpwzrYszJY+XtTlH7M55rbe\nZ7ufSIlBpSvyQNI1e5q7MMfdPnXdiOSlqAFB1U5zKcmKlNuaIJYeCI67farAKVLQlHbaFryNyxp4\nhvIRaZnKSEQpcZHt+VXGQSRvKO00n+RzWYNJkxqWqfBU6mlr5kjk8+cVka1SQChGUbN9XCUfRKSO\nAkImjS+EhXhhnDix+e310zOnTg3dROmCc1urOrqt9yJS0BQQGsumama+Srd52rSG1USnTWs6ezfq\nPIRi+F5EZJsUEOorli6UbCZqpT9jfY2DRrF8LyKybVFyU/Pl1SbzEIqpamacaxsX0/ciUmJQtdNW\n8BKqmplNNdZS+l5EikjUtFNNTGtsa10oua4/lJRJkxrOE0h3L2UaUC6l70WkBJXWGEJzWTL1+8ab\nW9qxmDQ3b6BUvxeREpPIE4KZ/RI4DvgEeAv4truvyelJo3SNmNWtBXDddeH9ddfB00+H7aV6J6yS\nFCIlIaknhP8DBrn7YOAN4LKcni1qloxnsWBMqYmyQI6IFLREnhDc/fF6b58HvprTE9a/o506Nbyg\n6QSsqPuVKpWkEClqiWcZmdmfgXvc/a7m9m11llHULBll04hIEUm8uJ2ZzTGzVzK8Tqi3z+XAJmDm\nNo4z1syqzax65cqVLW9QlAlY2ewnIlJsokxWyMUL+BbwHFAZ9W9aPDFNa/OKSAkjnxfIMbOjgB8A\nI9x9fRucMFqWjLJpRKSEJTKGYGZvAtsBq1Obnnf3c5r7u1jGEKIs3BL3fiIiCcrrmcru/v+SOG/k\nLJko+2VT8kFEpACU1kzluESd1yAiUkBUy6glNF9BRIpQ4vMQstFm1U6j0nwFESkAic9DKHqaryAi\nRUYBoSVU/VNEipDGEFpC8xVEpAhpDKE1NA9BRAqAxhDagqp/ikgRUUAQERFAAUFERFIUEEREBFBA\nEBGRFAUEEREBCizt1MxWAktzcOhewKocHLcQ6bsI9D0E+h6CQv8e+rl77+Z2KqiAkCtmVh0lR7cU\n6LsI9D0E+h6CUvke1GUkIiKAAoKIiKQoIAQ3J92APKLvItD3EOh7CErie9AYgoiIAHpCEBGRFAWE\nFDP7pZm9ZmYLzewBM+uedJuSYGYnm9m/zKzWzIo+q6IxMzvKzF43szfN7NKk25MUM/uNma0ws1eS\nbkuSzGxXM/urmS1K/X8xPuk25ZICQp3/Awa5+2DgDeCyhNuTlFeArwB/S7ohbc3MyoFfA0cDA4FT\nzWxgsq1KzG+Bo5JuRB7YBFzs7vsABwLnFfN/EwoIKe7+uLtvSr19HuiTZHuS4u6L3P31pNuRkM8B\nb7r72+7+CfAH4ISE25QId/8b8F7S7Uiau//X3een/v0hsAjYJdlW5Y4CQmZnALOSboS0uV2Ad+q9\nr6GI/+eX7JhZf+CzwAvJtiR3SmoJTTObA+yY4VeXu/tDqX0uJzwmzmzLtrWlKN9Dicq0wpHS8AQz\n6wzcB1zo7h8k3Z5cKamA4O5HbOv3ZvYt4EvA4V7E+bjNfQ8lrAbYtd77PsB/EmqL5Akza0cIBjPd\n/f6k25NL6jJKMbOjgB8Ax7v7+qTbI4l4EdjDzAaYWXvgFODhhNskCTIzA24DFrn7dUm3J9cUEOpM\nB7oA/2dmC8zsxqQblAQzO9HMaoDPA38xs9lJt6mtpJIKzgdmEwYP73X3fyXbqmSY2d3Ac8BeZlZj\nZmcm3aaEHAx8E/hC6rqwwMyOSbpRuaKZyiIiAugJQUREUhQQREQEUEAQEZEUBQQREQEUEEREJEUB\nQUqGmW2ulzq4IFWKINtjdDezc+Nv3Zbj721mz5nZRjP7Xq7OI5KJ0k6lZJjZR+7euZXH6A884u6D\nsvy7cnffHGG/HYB+wJeB99392pa0U6Ql9IQgJc3MylNrYbyYWgvj7NT2zmb2hJnNN7OXzSxd9fQa\nYPfUE8YvzWykmT1S73jTzez01L+XmNmVZvYMcLKZ7W5mj5nZPDOba2Z7N26Pu69w9xeBT3P+4UUa\nKalaRlLyOprZgtS//+3uJwJnAmvdfX8z2w541sweJ1Q9PdHdPzCzXsDzZvYwcClh3YyhAGY2splz\nbnD3Q1L7PgGc4+6LzewA4AbgC3F/SJGWUkCQUvJx+kJezyhgsJl9NfW+G7AHodDd1WZ2GFBLKIP9\nmRac8x7YUi3zIOCPoTwOANu14HgiOaOAIKXOgO+6e4OaTalun97AcHf/1MyWAB0y/P0mGna9Nt5n\nXepnGbAmQ0ASyRsaQ5BSNxsYlypxjJntaWadCE8KK1LB4H8IA70AHxKKIKYtBQaa2XZm1g04PNNJ\nUjX0/21mJ6fOY2Y2JDcfSaRl9IQgpe5WoD8wP1XqeCUhw2cm8GczqwYWAK8BuPtqM3s2tfj8LHe/\nxMzuBRYCi4F/buNco4EZZnYF0I6wROdL9Xcwsx2BaqArUGtmFwIDi3lRFskfSjsVERFAXUYiIpKi\ngCAiIoACgoiIpCggiIgIoIAgIiIpCggiIgIoIIiISIoCgoiIAPD/ASx8/HDDrR+DAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x58d25c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 权重初始化\n",
    "w = np.random.randn(3,1)\n",
    "\n",
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第 0次更新，分类错误的点个数：56\n",
      "第 1次更新，分类错误的点个数：23\n",
      "第 2次更新，分类错误的点个数：49\n",
      "第 3次更新，分类错误的点个数：46\n",
      "第 4次更新，分类错误的点个数：20\n",
      "第 5次更新，分类错误的点个数：26\n",
      "第 6次更新，分类错误的点个数：13\n",
      "第 7次更新，分类错误的点个数：22\n",
      "第 8次更新，分类错误的点个数：10\n",
      "第 9次更新，分类错误的点个数：19\n",
      "第10次更新，分类错误的点个数：10\n",
      "第11次更新，分类错误的点个数：17\n",
      "第12次更新，分类错误的点个数：10\n",
      "第13次更新，分类错误的点个数：17\n",
      "第14次更新，分类错误的点个数：10\n",
      "第15次更新，分类错误的点个数：15\n",
      "第16次更新，分类错误的点个数：10\n",
      "第17次更新，分类错误的点个数：13\n",
      "第18次更新，分类错误的点个数： 8\n",
      "第19次更新，分类错误的点个数：13\n",
      "第20次更新，分类错误的点个数： 6\n",
      "第21次更新，分类错误的点个数：13\n",
      "第22次更新，分类错误的点个数： 6\n",
      "第23次更新，分类错误的点个数：11\n",
      "第24次更新，分类错误的点个数： 4\n",
      "第25次更新，分类错误的点个数： 7\n",
      "第26次更新，分类错误的点个数：11\n",
      "第27次更新，分类错误的点个数：14\n",
      "第28次更新，分类错误的点个数： 4\n",
      "第29次更新，分类错误的点个数： 7\n",
      "第30次更新，分类错误的点个数：10\n",
      "第31次更新，分类错误的点个数：12\n",
      "第32次更新，分类错误的点个数： 5\n",
      "第33次更新，分类错误的点个数：10\n",
      "第34次更新，分类错误的点个数： 6\n",
      "第35次更新，分类错误的点个数：10\n",
      "第36次更新，分类错误的点个数： 6\n",
      "第37次更新，分类错误的点个数： 9\n",
      "第38次更新，分类错误的点个数： 5\n",
      "第39次更新，分类错误的点个数： 8\n",
      "第40次更新，分类错误的点个数： 5\n",
      "第41次更新，分类错误的点个数： 8\n",
      "第42次更新，分类错误的点个数： 6\n",
      "第43次更新，分类错误的点个数：10\n",
      "第44次更新，分类错误的点个数： 5\n",
      "第45次更新，分类错误的点个数： 7\n",
      "第46次更新，分类错误的点个数： 5\n",
      "第47次更新，分类错误的点个数： 8\n",
      "第48次更新，分类错误的点个数： 6\n",
      "第49次更新，分类错误的点个数：10\n",
      "第50次更新，分类错误的点个数： 5\n",
      "第51次更新，分类错误的点个数： 7\n",
      "第52次更新，分类错误的点个数： 5\n",
      "第53次更新，分类错误的点个数： 7\n",
      "第54次更新，分类错误的点个数： 6\n",
      "第55次更新，分类错误的点个数：11\n",
      "第56次更新，分类错误的点个数：13\n",
      "第57次更新，分类错误的点个数：10\n",
      "第58次更新，分类错误的点个数： 6\n",
      "第59次更新，分类错误的点个数： 9\n",
      "第60次更新，分类错误的点个数： 5\n",
      "第61次更新，分类错误的点个数： 7\n",
      "第62次更新，分类错误的点个数： 5\n",
      "第63次更新，分类错误的点个数： 6\n",
      "第64次更新，分类错误的点个数： 6\n",
      "第65次更新，分类错误的点个数：10\n",
      "第66次更新，分类错误的点个数： 6\n",
      "第67次更新，分类错误的点个数： 8\n",
      "第68次更新，分类错误的点个数： 4\n",
      "第69次更新，分类错误的点个数： 8\n",
      "第70次更新，分类错误的点个数： 4\n",
      "第71次更新，分类错误的点个数： 7\n",
      "第72次更新，分类错误的点个数： 5\n",
      "第73次更新，分类错误的点个数： 7\n",
      "第74次更新，分类错误的点个数： 5\n",
      "第75次更新，分类错误的点个数： 5\n",
      "第76次更新，分类错误的点个数： 6\n",
      "第77次更新，分类错误的点个数：11\n",
      "第78次更新，分类错误的点个数：12\n",
      "第79次更新，分类错误的点个数： 4\n",
      "第80次更新，分类错误的点个数： 7\n",
      "第81次更新，分类错误的点个数：10\n",
      "第82次更新，分类错误的点个数： 4\n",
      "第83次更新，分类错误的点个数： 5\n",
      "第84次更新，分类错误的点个数： 5\n",
      "第85次更新，分类错误的点个数： 5\n",
      "第86次更新，分类错误的点个数： 6\n",
      "第87次更新，分类错误的点个数：10\n",
      "第88次更新，分类错误的点个数： 5\n",
      "第89次更新，分类错误的点个数： 8\n",
      "第90次更新，分类错误的点个数： 4\n",
      "第91次更新，分类错误的点个数： 5\n",
      "第92次更新，分类错误的点个数： 5\n",
      "第93次更新，分类错误的点个数： 5\n",
      "第94次更新，分类错误的点个数： 6\n",
      "第95次更新，分类错误的点个数：11\n",
      "第96次更新，分类错误的点个数：11\n",
      "第97次更新，分类错误的点个数： 4\n",
      "第98次更新，分类错误的点个数： 7\n",
      "第99次更新，分类错误的点个数：10\n",
      "第100次更新，分类错误的点个数： 4\n",
      "第101次更新，分类错误的点个数： 5\n",
      "第102次更新，分类错误的点个数： 5\n",
      "第103次更新，分类错误的点个数： 4\n",
      "第104次更新，分类错误的点个数： 6\n",
      "第105次更新，分类错误的点个数：10\n",
      "第106次更新，分类错误的点个数： 5\n",
      "第107次更新，分类错误的点个数： 6\n",
      "第108次更新，分类错误的点个数： 5\n",
      "第109次更新，分类错误的点个数： 4\n",
      "第110次更新，分类错误的点个数： 7\n",
      "第111次更新，分类错误的点个数：10\n",
      "第112次更新，分类错误的点个数： 4\n",
      "第113次更新，分类错误的点个数： 5\n",
      "第114次更新，分类错误的点个数： 5\n",
      "第115次更新，分类错误的点个数： 5\n",
      "第116次更新，分类错误的点个数： 6\n",
      "第117次更新，分类错误的点个数：11\n",
      "第118次更新，分类错误的点个数：10\n",
      "第119次更新，分类错误的点个数： 4\n",
      "第120次更新，分类错误的点个数： 7\n",
      "第121次更新，分类错误的点个数： 8\n",
      "第122次更新，分类错误的点个数： 6\n",
      "第123次更新，分类错误的点个数： 6\n",
      "第124次更新，分类错误的点个数： 5\n",
      "第125次更新，分类错误的点个数： 4\n",
      "第126次更新，分类错误的点个数： 6\n",
      "第127次更新，分类错误的点个数：10\n",
      "第128次更新，分类错误的点个数： 9\n",
      "第129次更新，分类错误的点个数： 4\n",
      "第130次更新，分类错误的点个数： 7\n",
      "第131次更新，分类错误的点个数： 6\n",
      "第132次更新，分类错误的点个数： 4\n",
      "第133次更新，分类错误的点个数： 5\n",
      "第134次更新，分类错误的点个数： 5\n",
      "第135次更新，分类错误的点个数： 4\n",
      "第136次更新，分类错误的点个数： 6\n",
      "第137次更新，分类错误的点个数： 6\n",
      "第138次更新，分类错误的点个数： 4\n",
      "第139次更新，分类错误的点个数： 5\n",
      "第140次更新，分类错误的点个数： 5\n",
      "第141次更新，分类错误的点个数： 4\n",
      "第142次更新，分类错误的点个数： 6\n",
      "第143次更新，分类错误的点个数：10\n",
      "第144次更新，分类错误的点个数： 5\n",
      "第145次更新，分类错误的点个数： 5\n",
      "第146次更新，分类错误的点个数： 4\n",
      "第147次更新，分类错误的点个数： 5\n",
      "第148次更新，分类错误的点个数： 5\n",
      "第149次更新，分类错误的点个数： 4\n",
      "第150次更新，分类错误的点个数： 6\n",
      "第151次更新，分类错误的点个数： 6\n",
      "第152次更新，分类错误的点个数： 4\n",
      "第153次更新，分类错误的点个数： 5\n",
      "第154次更新，分类错误的点个数： 5\n",
      "第155次更新，分类错误的点个数： 4\n",
      "第156次更新，分类错误的点个数： 6\n",
      "第157次更新，分类错误的点个数：10\n",
      "第158次更新，分类错误的点个数： 5\n",
      "第159次更新，分类错误的点个数： 5\n",
      "第160次更新，分类错误的点个数： 4\n",
      "第161次更新，分类错误的点个数： 5\n",
      "第162次更新，分类错误的点个数： 5\n",
      "第163次更新，分类错误的点个数： 4\n",
      "第164次更新，分类错误的点个数： 6\n",
      "第165次更新，分类错误的点个数： 6\n",
      "第166次更新，分类错误的点个数： 4\n",
      "第167次更新，分类错误的点个数： 5\n",
      "第168次更新，分类错误的点个数： 5\n",
      "第169次更新，分类错误的点个数： 4\n",
      "第170次更新，分类错误的点个数： 6\n",
      "第171次更新，分类错误的点个数：10\n",
      "第172次更新，分类错误的点个数： 5\n",
      "第173次更新，分类错误的点个数： 5\n",
      "第174次更新，分类错误的点个数： 4\n",
      "第175次更新，分类错误的点个数： 5\n",
      "第176次更新，分类错误的点个数： 5\n",
      "第177次更新，分类错误的点个数： 4\n",
      "第178次更新，分类错误的点个数： 6\n",
      "第179次更新，分类错误的点个数： 6\n",
      "第180次更新，分类错误的点个数： 4\n",
      "第181次更新，分类错误的点个数： 5\n",
      "第182次更新，分类错误的点个数： 5\n",
      "第183次更新，分类错误的点个数： 4\n",
      "第184次更新，分类错误的点个数： 6\n",
      "第185次更新，分类错误的点个数：10\n",
      "第186次更新，分类错误的点个数： 5\n",
      "第187次更新，分类错误的点个数： 5\n",
      "第188次更新，分类错误的点个数： 4\n",
      "第189次更新，分类错误的点个数： 5\n",
      "第190次更新，分类错误的点个数： 5\n",
      "第191次更新，分类错误的点个数： 4\n",
      "第192次更新，分类错误的点个数： 6\n",
      "第193次更新，分类错误的点个数： 6\n",
      "第194次更新，分类错误的点个数： 4\n",
      "第195次更新，分类错误的点个数： 5\n",
      "第196次更新，分类错误的点个数： 5\n",
      "第197次更新，分类错误的点个数： 4\n",
      "第198次更新，分类错误的点个数： 6\n",
      "第199次更新，分类错误的点个数：10\n"
     ]
    }
   ],
   "source": [
    "for i in range(200):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))\n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        t = np.where(y != y_pred)[0][0]\n",
    "        w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$ ，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUFNW9L/Dvbx48BhAV0YhkZlAx\ncRhhGMDXUeAEBdRzjKj4uER8HryAMrjUpS5uAicJ5qFLI3J8jK9zspjkSnwdNSKEqBEUcx1g8KAY\n8QVOVEQiD0EQpn/3j901/Zjumeruqt7V1d/PWrWG6ump2l0x9eva39q7RFVBRERUYrsBREQUDCwI\nREQEgAWBiIiiWBCIiAgACwIREUWxIBAREQAWBCIiimJBICIiACwIREQUVWa7AZk47LDDtLq62nYz\niIgKyurVq79U1f5dva+gCkJ1dTWam5ttN4OIqKCIyCY372OXERERAWBBICKiKBYEIiICUGAZQir7\n9+9Ha2sr9u7da7spBalHjx4YOHAgysvLbTeFiCwr+ILQ2tqKPn36oLq6GiJiuzkFRVWxbds2tLa2\nYtCgQbabQ0SWFXyX0d69e9GvXz8WgyyICPr168erKyICEIKCAIDFIAc8dkTeaWoCqquBkhLzs6nJ\ndosyU/BdRkREQdDUBEybBuzZY9Y3bTLrADBlir12ZSIUVwi2lZaWoq6uDrW1tZg8eTL2OP9FZOCa\na67BO++8AwC4/fbbE3536qmnetJOIvLPnDmxYuDYs8e8XihYEDzQs2dPtLS0YP369ejWrRseeOCB\njLfx8MMPo6amBkDHgvD666970k4i8s/mzZm9HkRFVxD87uM7/fTT8f777wMA7rrrLtTW1qK2tha/\n+c1vAAC7d+/GOeecg2HDhqG2thaPP/44AGDs2LFobm7Grbfeim+++QZ1dXWYEr3O7N27NwDg4osv\nxgsvvNC+ryuuuAJPPvkk2tracPPNN2PUqFEYOnQoHnzwQW8/FBF1qbIys9cDSVULZhkxYoQme+ed\ndzq8ls6iRaoVFapAbKmoMK/nolevXqqqun//fj333HP1vvvu0+bmZq2trdWvv/5ad+3apTU1Nbpm\nzRp94okn9Jprrmn/2+3bt6uq6pgxY/TNN99M2F7y9p966imdOnWqqqru27dPBw4cqHv27NEHH3xQ\nf/azn6mq6t69e3XEiBH64Ycfum5/JseQiFLz6/ziBQDN6uIcW1RXCH718Tnf6EeOHInKykpcffXV\nWLlyJSZNmoRevXqhd+/eOP/887FixQqccMIJWL58OW655RasWLECffv2db2fs846Cy+99BL27duH\nJUuWYPTo0ejZsyeWLVuG3/72t6irq8NJJ52Ebdu2YePGjbl9KCLKyJQpQGMjUFUFiJifjY2FEygD\nRXaXkV99fE6GEM8U5Y6OO+44rF69Gi+88AJuu+02jB8/Hj/5yU9c7adHjx4YO3Ysli5discffxyX\nXnpp+77uvfdeTJgwIbcPQkQ5mTKlsApAsqK6QshnH9/o0aPxzDPPYM+ePdi9ezeefvppnH766fj0\n009RUVGBH/3oR7jpppuwZs2aDn9bXl6O/fv3p9zuJZdcgsceewwrVqxoLwATJkzA/fff3/437733\nHnbv3u39hyKiUCuqK4T58xPvEwaAigrzutfq6+txxRVX4MQTTwRgbisdPnw4li5diptvvhklJSUo\nLy/H/fff3+Fvp02bhqFDh6K+vh5NSan3+PHjMXXqVJx77rno1q1b+7Y//vhj1NfXQ1XRv39/PPPM\nM95/KCIKNUnXtRFEI0eO1OQH5GzYsAHHH3+86200NZnMYPNmc2Uwf35hX+J5IdNjSESpzZhhcoO2\nNqC01HwBve8+260CRGS1qo7s6n1FdYUAFH4fHxEF04wZQPwFf1tbbD0IRcGNosoQiIj80tiY2etB\nxIJAROSBtrbMXg8iFgQiIg+Ulmb2ehCxIBARdcHNlDfOzKZuXw+ioguViYgy4XZaayc4DuJdRm5Z\nu0IQke+KyMsiskFE3haRBlttyZWI4MYbb2xfv/POOzFv3jzP98NpsYnyL5Mpb+67DzhwwMxkdOBA\nYRUDwG6X0QEAN6rq8QBOBjBTRGp832vyuAsPxmF0794dTz31FL788suct9UZTotNlH9hmNbaLWsF\nQVU/U9U10X/vArABwFG+7nTePOCGG2JFQNWs5/htvqysDNOmTcPdd9/d4Xdbt27FBRdcgFGjRmHU\nqFF47bXX2l8/88wzUV9fj2uvvRZVVVXtBeW8887DiBEjMGTIEDRG71njtNhEdoRiWmu33EyJ6vcC\noBrAZgAHpfjdNADNAJorKys7TOvqeurmSES1ocHMSdvQkHo9S7169dIdO3ZoVVWVbt++Xe+44w6d\nO3euqqpeeumlumLFClVV3bRpk37/+99XVdWZM2fq7bffrqqqS5YsUQC6detWVVXdtm2bqqru2bNH\nhwwZol9++WX7fpL3q5r7tNic/poovSBPa+0WXE5/bT1UFpHeAJ4EMFtVdyb/XlUbATQCZuqKHHYE\nON/g77nHLADQ0GBez/Fh8wcddBCmTp2KBQsWoGfPnu2vL1++vP3RmACwc+dO7Nq1CytXrsTTTz8N\nAJg4cSIOOeSQ9vcsWLCg/XeffPIJNm7ciH79+qXd91lnnYVZs2Zh3759ePHFFxOmxX7rrbfwxBNP\nAAB27NiBjRs3YtCgQTl9VqJi4gTHxTDljdWCICLlMMWgSVWfysMOzcnfKQaAJ8XAMXv2bNTX1+PK\nK69sfy0SiWDVqlUJRQJIPz32K6+8guXLl2PVqlWoqKjA2LFjsXfv3k73y2mxifxVLFPe2LzLSAA8\nAmCDqt6Vl506mUG8+EwhR4ceeiguuugiPPLII+2vjR8/HgsXLmxfd56bcNppp2Hx4sUAgGXLluGr\nr74CYL7FH3LIIaioqMC7776LN954o/1vOS02EfnJ5l1G/wTgMgA/EJGW6HK2b3tzisE995huokjE\n/LznHk+Lwo033phwt9GCBQvQ3NyMoUOHoqamBg888AAAYO7cuVi2bBnq6+uxZMkSHHnkkejTpw8m\nTpyIAwcOYOjQofjxj3+Mk08+uX1bzrTYU1J8VRk/fjxeffVVnHHGGQnTYtfU1KC+vh61tbW49tpr\nceDAAU8+JxGFT3FNfz1vHrB9e6ybyCkSBx+c851Gmdq3bx9KS0tRVlaGVatWYfr06R2eupYvnP6a\nKJi8mq6f01+nMm+eKQJOZuBkCh5lCJnYvHkzLrroIkQiEXTr1g0PPfRQ3ttARMHldoS0l4qrIAAd\nT/4WigEADB48GGvXrrWybyIKvs5GSPtVEEIxuV0hdXsFDY8dUTDZGCFd8AWhR48e2LZtG09sWVBV\nbNu2DT169LDdFCJKYmOEdMF3GQ0cOBCtra3YunWr7aYUpB49emDgwIG2m0HkSjE9E33+/MQMAQAq\nKszrfin4glBeXs6Rt0RFwEbIapONEdIFf9spERWH6mpTBJJVVQEff5zv1hQWt7edFnyGQETFoZim\nobaFBYGICkJRTUNtCQsCERWE+fNNqBrP75C12LAgEFFBmDLFPK+4qsqMJ62qMuthDJRtKfi7jIio\neBTLNNS28AqBiIgAsCAQEVEUCwIRWdfUZMYZlJSYn01Nwdpe0PfrFWYIRGSV1yOQbY1oDsNIao5U\nJiKrvB6BbGtEc5BHUnOkMhEVBK9HINsa0RyGkdQsCERkldcjkG2NaA7DSGoWBKJOFHpIWAjmzwfK\nyxNfKy/PfgSyrRHNYRhJzYJAlIYTEm7aZB7F7YSELAre8/LJtrZGNIdhJDVDZaI0ghwShgmPs/8Y\nKhPlKAwhYSHgcQ4OFgSiNMIQEhYCHufgYEEgSiMMIaEjyCN3w3ScC56qFswyYsQIJcqnRYtUq6pU\nRczPRYtstyhzixapVlSommjcLBUV/n+WTPYbhuMcZACa1cU5lqEyUchx5C4xVCYiABy5S+6xIBCF\nHEfuklssCER5ZGOa50IYucsR4QHhJmgIysJQmQqZ1+FuIYS2bvZrK/QuJmCoTBQsYZnm2Wth+RxB\nVhChsog8KiJfiMh6m+0gyoewTPPstbB8jjCwnSH8J4CJlttAlBdhmebZa2H5HGFgtSCo6qsA/mGz\nDUT54nW4O3++CWHjlZTkFhbbCHc5Ujk4bF8hEBUNr6dHfu01IBJJfC0SMa9nw9Z032GYNjosrIfK\nIlIN4HlVrU3z+2kApgFAZWXliE2p0ieiIlRWBrS1dXy9tBQ4cCDz7THcDa+CCJXdUNVGVR2pqiP7\n9+9vuzlEgZGqGHT2elcY7lLgCwIRpVZamtnrXWG4S7ZvO/09gFUAvicirSJytc32UGHzIxCdMcN0\nzYiYnzNm5PY+L02bltnrXWG4S9ZHH2eycKQypePHaNfp0xO35yzTp2f3Pq8tWqRaWpq4z9LS3D4z\np6EOJ3CkMhUTPwJRt6Gt1+GuWwyBya3QhMpEbvgRiLoNbb0Od91iCExeY0GgUPAjEHUb2nod7rrF\nEJi8xoJAoeBHIOo2tM0k3PUy+Ob00uQ5N0FDUBaGytQZPwLR6dNjwW1paeqgeNEi1bKyxHC3rKzj\n/v0Ivjm9NLkBhspE+eE23OWzjckWhspEeeI23OWzjSnoWBCIcuQ23OWzjSnoWBCo6HgdsLoNd/ls\nYwo8N0FDUBaGypQrvwJWt4E2n21MNoChMlFHDFjT47EJL4bKRCkwYE2Px6YTe/fabkFepC0IInKC\niLwhIp+ISKOIHBL3u/+Xn+YReYsBa3o8NjCPnNu4EVi8GLjtNmDiROA73wF+9CPbLcuLzq4Q7gcw\nD8AJAN4DsFJEjon+rtzndhH5wq9g18b0114ruumv9+0D1qwBHn0UuP564LTTgL59geOOAy6+GLjz\nTuDzz4GzzgLOO892a/OirJPf9VbVF6P/vlNEVgN4UUQuA1A4wQNRHOc5vXPmmK6Qykpzwsvl+b0z\nZgD33x9bb2uLrd93X/bbzTc/jk1g7NwJtLSYZe1as7zzDrB/v/l9797AsGHAFVcAw4ebpaYG6N7d\narPzLW2oLCLrAIxW1R1xrw0F8CSAQ1W1X36aGMNQmYLI1vTXlMZnn8VO+k4B+OCD2O8PPzx20neW\nY44x99qGlNtQubMrhF8BOB7AG84LqvqWiIwD8OPcm0gUDramvy56kYg50Tsnf6cAbNkSe88xx5gT\n/pVXxk7+Rx5pr80Bl7YgqOrv0ry+GcC/+dYiogJTWpr+CoE88u23wNtvJ578160Dvv7a/L6sDBgy\nxPT3Dx8O1NWZLqC+fe22u8B0doVARC5Mm5aYIcS/nqypyds+eq+3Fwg7d5qTffzJv7P+/ro6UwyK\nrL/fDywIRDlyguPGRnOlUFpqikFyoNzUZF7fs8esb9oUKxrZnMS93p4VTn9/fNibqr/f+eZfBP39\nNnGkMlGeeD0SuKBGFsf398ef/JP7++vqEsNe9vd7wotQ2dnQcTBjEo5Q1dronUbnqurPPWgnUdHw\neiRwYEcWJ/f3t7SYLqBdu8zvnf7+iRNjJ3729weCmy6jhwDcDOBBoP1Oo98BYEGg/FA1I77SrReI\nysrU3+izHQns9fay4ra///LLY9/+2d8fWG464ipUNXmqCt5dTWmdcYY5XzvLGWekfp+rqZbnzQNu\nuMEUAcD8vOEG83oeeP0M5G7dEl/r1i37kcB+jCzu9PN+/jmwZAlw++3A5MnAsceab/WjRwMNDcAL\nL5hpHm68EXj8ceBvfwN27ABWrgTuvRe4+mqgvp7FIMi6mg4VwBIAxwBYE12/EMASN1Oper1w+uvg\nGzcucfpkZxk3LvF9rqZajkRUGxrMLxsaUq/7yOvpoBctUi0vT9xeebn/z1TOZFsVFaqCNj0W7+mF\nWKy/LrtN/z50ouoRRyQ2/OijVS+4QPXnP1f94x9V//533//3oOzBq+mvReRoAI0ATgXwFYCPAExR\n1RQXq/5iqBx8nfXkxP+n5joQda4I7rkn9lpDA3D33b53GxVFCBzX3//Y7BYcu2sthmEdDoLp79+P\nMmwsH4Ka/1XH/v4C5jZU7rQgiEgJgAtVdbGI9AJQoqq7PGxnRlgQgs9tQSgpSVyP//tIJMUfxt9m\nGInkJUPIqI0Wtpex+P5+506ft99u7+//Gr3Qgjq0oA5rMRxrMRxvYwj2S/f8tI9848ldRqoaEZHr\nACxW1d2etY6KnutA1LlCiHfDDXm5QiiIEDhd4P755x2ndHj//dj7nPv7J0xo/+Z/wpnH4uPNHWPF\nqmKa/rrYddWnBDNv0U0AvgvgUGdx0x/l9cIMwS43/dV+ZQiP9GlQQUQf6ZPfDMHLPn/PH1E5d67q\nrFmq772nunix6q23mv9hevdO39///PNp+/v5CM3wgssMwU1B+CjF8qGbjXu9sCDYk8nJIrkoJBeD\n+G12VWDWTZqrC8saFIhEtxfRhWUNum7SXA8/Xfr2deuW+Fm6dbMYAu/bp7p2reqjj6ped53qgAGJ\njSspMT+PP1717rtV//IX1e3b89c+Ciy3BYEjlckVW4Go2a8CiO8eUlRVie9BrNUQ2Onvjx/VG9ff\nj169TLj79dfAW2/F/i5PgTsVFk9C5eiGpqZ6XVV/m2XbssaCYI+tQNRmEJu3fTv9/fEn/1T9/fHT\nOhx7bKyBFgJ3KiyeTV0BYFTcv3sAGAdgDYCcC4KITARwD4BSAA+r6i9z3Sb5w9aoWJujcc2+O16d\nVFZmecKNRIAPP+z48JbPP4+95+ijzQn/8ssT5/NJdZJXHwJ3DceocMqSm36l+AVAXwDPZvp3KbZT\nCuADAEcD6AZgHYCazv6GGYI/3PQb2wocM9mv1/3f6ybN1QUlifnFghKX+UV8f//116uedppqnz6x\nD1FWpjp0qOrll5v+/ldeyay/349Be3PnJv6ts825Lj4vBRq8CpU7/AFQDmBDpn+XYjunAFgat34b\ngNs6+xsWBO/ZPOFm0sa8F6xIRDdMMCfYu2GKwt0w6xsmJJ1wd+5UXbFCdcEC1SuvVK2rS7w9qVcv\n1VNPVZ05U/Xhh1Wbm1W/+SbLhsXx8gRueVQ4+cttQXCTITwHwHlTCYAaAH9Q1VtyuTIRkQsBTFTV\na6LrlwE4SVWvS/c3zBC8F8jRs1nw43NUVylmb74BsxEbJd2Ia/Ba//PxXzek6e/v3z/183r9enya\nl108TheUhVHh5C8vQ+UxcasHAGxS1dYc2wcRmQxgQlJBOFFVr0963zQA0wCgsrJyxKZU/6+nrFkf\nPesRzz9HJILBpR+iDmvwB1yc+j1HH516/v5CPnkypA4lL0Pls5OvBkTkV7leIQBohRns5hgI4NPk\nN6lqI8xcShg5cmTh3CNbIKxOoezht9ucPse335opm+OD3pYWbETHWVr+gtG47/B/x+N/qwMOPjir\ntlrh5lg7Vwjx8jQqnILBzfTXZ6Z47SwP9v0mgMEiMkhEugG4BMCzHmyXMuDHFMquZDCttZspqF1/\njl27sOwnKzH30HvxmFyF9d3r0VbR23y7v+oq4JFHzHMwL7sMnw0ZBwC4FzMhiOA3aMAYvIp/H/5M\nYU3u5uZYx3cXNTSYK4OGBrMe/7cUbunCBQDTAfwPgN0A3opbPgKwyE1A0dUC4GwA78HcbTSnq/cz\nVPZH3sPiDALMXELvJ/7jc9UlS1Rvv1118mTVwYM1ItK+oS3ory9ivN5ZdouumPl71XffVT1wILbB\nuXN1w4QGraqMmG1WRoPmQrrrJpOwmHcZhRZyDZVFpC+AQwD8AsCtcb/apar/8KM4dYWhcoi4DDBd\nhcXO/f3xA7uS7+8fNAgYPhx3/nk4Xt4xHC2ow6cYAGeMQdrw2cvQ1pZMwuIwfF7qwLNQOW6Dh8MM\nTAMAqGren9zKghAyLgLM5LC4HN+iBu9gOFrw2Ky4AV7O83pLS4GamsSgd9iw9v7+sIToGWNYXNQ8\nC5VF5F8B3AVgAIAvAFQB2ABgSK6NpCLmJsDctQvnH74O39nSEp2dfy2G4G10x7fm949E5/O57LLY\nif+EE4Ae0e8tkUjiSRBZhM9B/8ZsKywO+nGh7HTVpwQzgrgfgLXR9X8G0OimP8rrhRlCSKTqx77m\nGrN+6qmqF12kOniwCQOS+vt/gVt0arff63//Oqm/f8wYMyCsrc2st7WZ9TFjEnad0QC2oPepu2kf\nRzSTus8Q3NxltF9VtwEoEZESVX0ZQJ1P9YnCzunv//RTYNQo4L33gKOOAh5+2Pz+9deBN9803/R/\n+lPguefw1IJWnFi5BWfJUjxQ9UuMf/QSnHvz92KDvSIR8zD3lhZgxAizPmKEWd+xI6EvaMoUoLHR\nZAYi5mdjo3k9gSqwfXviXTZOP/z27fbvunHbPhHTXRafGdx9t1k/+ODMv9UH/bhQbrqqGACWA+gN\nYCGA38NMRve6m2rj9cIrhAKzb59qS4vqY4+ZB7mcfrrqQQfFvpqXlqqecILq1KlmPp+XXlL96qvs\n9uVcEcR/9Y+/YshG/LdpZwnSNA6ZtC/5tVw+Q9CPC3UAD6eu6AXgG5gxC1NgJrdrUnPVkFcMlQNs\n167U8/d/G+3vr6gwffzxYe+QIbH+fi9EIolTRLS1dcgQMqYBD2NttS/ox4USeBYqq+puEakCMFhV\n/0tEKmBmKqVitWVL6vn7nS8Xhx1mTvizZyfO3+/XfD6AOSHV1ye+Vl8PrFmTfVFwukPi5SuMdfM+\nP9rnhq39kv+6uoQA8G8wo4o/iK4PBvBnN5cfXi/sMsqzSET1/fdV//AH1TlzVM8+W/XIIxO7CgYN\nUj3/fNWf/lT1uedUW1vz33XQ1qZ6xBGmPcOGmfVhw8z6EUdk121kM4y1FRa7wVlRCxJcdhm5mcto\nJoATAfw1WkA2RsckUJjs3x+bz8f59t/SYh7lCMTu7z/zzNjTu+oCMp+PCFAW/U95zBizPmaM6cIq\nK8vuW2u6MBbIPYwFzLbip4pwrgDcvs/r9rlla7+UH11VDAB/jf50bjstA/CWm2rj9cIrBI/s2qW6\ncqXqvfeqXnWV6vDhiU+Tr6hQPeUU1RkzVB96SPXNN72Zv99PkYgJruOvXmbNyv0bq40w1lZYnAlb\n+6WswMNQ+dcAtgOYCuB6ADMAvKOqc/wrU6kxVM7Cli0dp3RI1d/vLHV1wODB/vb3+0ULIOh028ZC\n+CxUMLyc/vpWAFfDTHR3LYAXADycW/MoJ063Qfw6kHo+n88+i71v0CBzwndG9g4fDgwYYH6Xantu\n9puPk1TyiOMUI5AzCjr9+Bxutum2jZl8FiIvpbt0AFDp5hIjn0uuXUa2HgHpqblzVa+7zjyv17m/\n/6ijVLt3j3UvOPf3X3aZ6l13qb78cvr7+70MOv3gZgSy7Rk9vQyBGdqSD+BBqPwMgHoAEJEnVfWC\nPNQn3zQ1AdOmAXv2mPVNm8w6kGKUapB8/bUJR51v/M8/D3zxBbBwofl9WRlw4AAwdCgwc6a51bK2\n1t39/epx0Om15BHIq1fHRiDX1cWuFNwGnX58DrfbdNtGhrZkU7pKgWiInPxvm0suVwhVVbEv0PFL\nVVXWm/Teli2qL76o+otfmPl8jjsuYT4fPeww1TPPVB0xIvFD5PLN0Y+g00uZjEB2E3T68Tn8CIEZ\n2pKH4MHzENaoan3yv23KJVQO1LTHqsBHHyX29be0mPl9HNXVHcPeo46KfWP3MnB0uz2v9+uW1yOQ\n/fgcto4NkQtehMrDRGQnzBNEekb/jei6qupBHrQzb6w9O9i5vz8+7E2+v//444Fx4xLn7z/kkNTb\nU4/DU7fby2S/XnImqovndB9lUxQy/RxeHsNM2+nViGYit9xcRgRlyaXLKKNpj7Pl3N+/cKHq1Ver\n1td3vL//5JNVp09XbWzM/P5+r8PToAed8d1FTjdR8nomMv0cQZ9emtNQk0vwcKRyKDjB8Zw5wObN\n5spg/vwcAuUvvkj8xr92LbBxY8f7+xsaYt/8c72/3+vwNOhBZ0mJeZh9XV3sisAJlvv2zfwKIZPP\n4fUxdMvtft2+jygTbqpGUBYrI5UjEdUPPlB94gkzn88556gOGJB4qVFdrTppkpnP59lnVT/5xN8Q\n0OvwNOhBZ/KVQC5TWqtm9nmDPL00p6Eml+DVSOUg8X2k8v79wIYNHcPe5P7++KC3ri59f79tyqAz\nZ7aOodv98n9jcsHLkcrhtm4dcO+95uS/fn3i/P1Dh5o+pfj5+3v2tNtet5SjXdNK7k5J172SyTF0\nu0237Qty0E/h5eYyIiiLL11GL7+s2q+f6hlnqN58s+rvfqe6YUPi83oLDUe7puc2iLU1+jnoQT8V\nJDBUdmn0aGDr1nB9o+Jo19Q0gyDW1ujnoAf9FGrMEMLMy26MsHC6WZwTOJB4Uk31frfjENxu0207\nOQ6BPOI2Q2BBoOLjRxDLcJcCzG1ByPEJ5BQKyV8K0n1JSJ7jI9c5P9zu10vpgthc9u3HNoksYEEo\ndvPmJZ68nJPbvHmJ7xs71gwIc4qAM6XE2LH+7tdL8V07DQ3mMzQ0mPVsT+B+bJPIEhaEYhYfiDon\nL+fktn177GSWPA21UwxaWszrmV4puN2v19IFsQ0N2QexfmyTyBJmCMXObSAaXwQc8VNK+LVfP/gR\nxDLcpQBjqEzuuQ1EC2EaaiLqgKEyuaMKzJ6d+Nrs2R27bSIR8zS2ePX1qbuL3ITFDGKJAsdKQRCR\nySLytohERKTLqkU+UQVOOQVYsACYNcuc3GfNMuunnJKYIQwYYKb5GDbMXBkMG2bWBwxILApuwmIG\nsUSBZOsKYT2A8wG8amn/lAkR8+xmABgzxqyPGWPWy8pSj9rtLCxmEEsUTG7mt/BrAfAKgJFu329l\n+uuwi0RUZ83ShCmUZ81KPdWy2/fZmDKaiNJCIUx/LSKvALhJVV0lxQyVfeI23PX6fUSUF9ZDZRFZ\nLiLrUyw/zHA700SkWUSat27d6ldzw8nNyGK34W4m73MTUmfC7YhmGyOficLEzWWEXwvYZeSfMWMS\nnzvsPI94zJjYe7yeajkSUT3ppMTuJKeb6aST+HxhIkvgssuIt52GkduRxW7DXVshsLoMqd2+j4g6\n56ZqeL0AmASgFcA+AFsALHVkHY0WAAAIi0lEQVTzd7xCyIBzRRAf7sZfMcTz8pnKbsNnt/h8YaKc\noRBC5UwxVM6Q1yOL3VKPQ2W32/N6v0QhYT1UpjjJRTcfRdjpJooXP1tpZ+3JpX3qcajsdP/EyyX0\nJqK0WBD8ZmOa5/jMoK7OXBnU1SVmCn60T12OfM5ke25GNLt9HxF1is9U9lN82Ank/rxdt0pKgL59\nE2cjXb3aFIO+fWPdKrba5xafL0yUX26ChqAsBRkq2ww7kwPkdIGyl+3zOlR2ttnZeqbvIyoyYKgc\nIEEPO22FwESUFwyVg8Jm2OkmLPa6fQx3iQoWC4KfbIadNqahZrhLVNAYKvvJVtjpNiz2un0Md4kK\nGjOEfEi+Wycfd+/Ef1t3pHtmsdfts/F5iSgtPlOZGO4SEQCGyuT1iGEiCj0WhDDyesQwERUFFgQi\nIgLAghBOIsCqVbGrgpKS2NXCqlXMEYgoJYbKYcZQmYjAUJkyGTFsY3puIgocFoQwymTEsI3puYko\nkDhSOYzcjhgO+vTXRJRXzBDCzM2I4UxGNBNRQeJIZXKP4TNRqDFUzkUxhawMn4koigUhWTGFrAyf\niSgOC0K8+JDVOfk5J8zt28P3jThd+NzQkD58LobjQlSkmCEkK8aQleEzUagxVM4FQ9bUeFyIChJD\n5WzxmcCp8bgQhR4LQjw+Ezg1HheiosCRyvH4TODUeFyIigIzhFT4TODUeFyIChIzhFwkn+R40jN4\nXIhCrbgKgtuRthyRS0RFqHgKgtuRthyRS0RFykpBEJE7RORdEXlLRJ4WkYN93aHbkbYckUtERcxK\nqCwi4wG8pKoHRORXAKCqt3T1dzmFym5H2nJELhGFTMGMVBaRSQAuVNUpXb0357uM3I605YhcIgqR\nQrrL6CoAS3zfi9uRthyRS0RFyreCICLLRWR9iuWHce+ZA+AAgKZOtjNNRJpFpHnr1q3ZNcbtSFuO\nyCWiIubbSGVVPaOz34vI5QD+BcA47aTfSlUbATQCpssoq8a4HWnLEblEVMRshcoTAdwFYIyquv7a\n70mG4GakLUfkElGIBD1DWAigD4A/iUiLiDyQl726HWnLEblEVISsTG6nqsfa2C8REaUXhLuMiIgo\nAFgQiIgIAAsCERFFsSAQEREAFoTccJpsIgoRFoRscZpsIgoZFoRscJpsIgohK+MQCl78lBb33BOb\nKpvTZBNRAbM+/XUmcp66wmucJpuICkDQp64ofJwmm4hChgUhG5wmm4hCiBlCNjhNNhGFEDOEXHCa\nbCIqAMwQ8oHTZBNRiLAgEBERABYEIiKKYkEgIiIALAhERBTFgkBERABYEIiIKKqgxiGIyFYAm3zY\n9GEAvvRhu4WIx8LgcTB4HIxCPw5Vqtq/qzcVVEHwi4g0uxm0UQx4LAweB4PHwSiW48AuIyIiAsCC\nQEREUSwIRqPtBgQIj4XB42DwOBhFcRyYIRAREQBeIRARURQLQpSI3CEi74rIWyLytIgcbLtNNojI\nZBF5W0QiIhL6uyqSichEEfmbiLwvIrfabo8tIvKoiHwhIuttt8UmEfmuiLwsIhui/79osN0mP7Eg\nxPwJQK2qDgXwHoDbLLfHlvUAzgfwqu2G5JuIlAL4DwBnAagBcKmI1NhtlTX/CWCi7UYEwAEAN6rq\n8QBOBjAzzP9NsCBEqeoyVT0QXX0DwECb7bFFVTeo6t9st8OSEwG8r6ofquq3AP4vgB9abpMVqvoq\ngH/YbodtqvqZqq6J/nsXgA0AjrLbKv+wIKR2FYAlthtBeXcUgE/i1lsR4v/zU2ZEpBrAcAB/tdsS\n/xTVM5VFZDmA76T41RxV/e/oe+bAXCY25bNt+eTmOBSpVI+84214BBHpDeBJALNVdaft9vilqAqC\nqp7R2e9F5HIA/wJgnIb4ftyujkMRawXw3bj1gQA+tdQWCggRKYcpBk2q+pTt9viJXUZRIjIRwC0A\nzlXVPbbbQ1a8CWCwiAwSkW4ALgHwrOU2kUUiIgAeAbBBVe+y3R6/sSDELATQB8CfRKRFRB6w3SAb\nRGSSiLQCOAXAH0Vkqe025Uv0poLrACyFCQ8Xq+rbdltlh4j8HsAqAN8TkVYRudp2myz5JwCXAfhB\n9LzQIiJn226UXzhSmYiIAPAKgYiIolgQiIgIAAsCERFFsSAQEREAFgQiIopiQaCiISJtcbcOtkSn\nIsh0GweLyAzvW9e+/e+LyCoR2SciN/m1H6JUeNspFQ0R+VpVe+e4jWoAz6tqbYZ/V6qqbS7edziA\nKgDnAfhKVe/Mpp1E2eAVAhU1ESmNPgvjzeizMK6Nvt5bRP4sImtE5H9ExJn19JcAjoleYdwhImNF\n5Pm47S0UkSui//5YRH4iIisBTBaRY0TkRRFZLSIrROT7ye1R1S9U9U0A+33/8ERJimouIyp6PUWk\nJfrvj1R1EoCrAexQ1VEi0h3AayKyDGbW00mqulNEDgPwhog8C+BWmOdm1AGAiIztYp97VfW06Hv/\nDOB/q+pGETkJwH0AfuD1hyTKFgsCFZNvnBN5nPEAhorIhdH1vgAGw0x0d7uIjAYQgZkG+4gs9vk4\n0D5b5qkA/mCmxwEAdM9ie0S+YUGgYicArlfVhDmbot0+/QGMUNX9IvIxgB4p/v4AErtek9+zO/qz\nBMD2FAWJKDCYIVCxWwpgenSKY4jIcSLSC+ZK4YtoMfhnmKAXAHbBTILo2ASgRkS6i0hfAONS7SQ6\nh/5HIjI5uh8RkWH+fCSi7PAKgYrdwwCqAayJTnW8FeYOnyYAz4lIM4AWAO8CgKpuE5HXog+fX6Kq\nN4vIYgBvAdgIYG0n+5oC4H4R+T8AymEe0bku/g0i8h0AzQAOAhARkdkAasL8UBYKDt52SkREANhl\nREREUSwIREQEgAWBiIiiWBCIiAgACwIREUWxIBAREQAWBCIiimJBICIiAMD/B84mruk8YZxcAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9f2c860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算分类正确率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.95\n"
     ]
    }
   ],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)\n",
    "loc_n = np.where(s < 0)[0]\n",
    "y_pred[loc_n] = -1\n",
    "accuracy = len(np.where(y == y_pred)[0]) / len(y)\n",
    "print('accuracy: %.2f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PCA是机器学习最简单的算法之一。PCA处理线性可分问题，优化的PCA解决线性不可分的问题。实际验证表明，一般的PCA处理线性可分及线性不可分问题都有不错的表现，即一般能得到最佳的分类直线。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
